From Alan.Bateman at oracle.com Mon May 1 11:41:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 May 2017 12:41:30 +0100 Subject: Dependencies to a root modules when creating a dynamic module layer In-Reply-To: <1683561662.252290.1493632110367.JavaMail.zimbra@u-pem.fr> References: <1683561662.252290.1493632110367.JavaMail.zimbra@u-pem.fr> Message-ID: On 01/05/2017 10:48, Remi Forax wrote: > Hi all, > i try to load a kind of dynamic plugin using a ModuleLayer, > the plugin is wrapped in a module which is loaded dynamically. > This module has a requires directive to "java.sql" which is part of the 'root' modules. > > The problem is that my application do not requires "java.sql" because it do not need it, > so "java.sql" is not part of the module loaded when the VM is booted, > then when i load the plugin module > - either it fails to find the required module java.sql [1] > - or i try to load java.sql from the ModuleLayer and get an exception because java.sql can not be loaded by the boot classloader > > > I'm sure i've missed something ? > There isn't general support for dynamically augmenting the platform modules. There is partial support to allow the jdk.management.agent and java.instrument modules be dynamically loaded but that's all at this time. One thing to point out is `--add-modules ALL-DEFAULT` in JEP 261. This is motivated by the scenario where a container needs to load applications that require additional modules. -Alan. From stephen.felts at oracle.com Mon May 1 11:48:31 2017 From: stephen.felts at oracle.com (Stephen Felts) Date: Mon, 1 May 2017 04:48:31 -0700 (PDT) Subject: jake -> jdk9/dev In-Reply-To: <29b55d14-a38b-c7e4-10c2-d9259ef9a7a9@oracle.com> References: <408902ef-689b-8dfa-2750-8ef629244910@oracle.com> <29b55d14-a38b-c7e4-10c2-d9259ef9a7a9@oracle.com> Message-ID: <06671bc1-3c63-4622-9a91-e8cef507a79d@default> I think this statement is very misleading. As part of this "small changes", jdk.attach.allowAttachSelf will be set to false and everyone that has self-attaching agents will start failing. So to repeat myself, I think that the Java community made it clear that this feature should not be included in JDK9. -----Original Message----- From: Alan Bateman Sent: Saturday, April 29, 2017 7:46 AM To: Stephen Felts; jigsaw-dev Subject: Re: jake -> jdk9/dev On 29/04/2017 11:04, Stephen Felts wrote: > I think that the Java community made it clear that this feature should > not be included in JDK9, including going to the effort of writing a > paper to that effect at > https://docs.google.com/document/d/19H1iGUnyI4Y40U5sNVAmH0uLxcpNfYSjPI > W3s0Rv5Ss/edit?usp=sharing > This is the proposal for the EnableDynamicAgentLoading option. The default is true so that tools can load agents into target VMs as before. The proposal is that some future version disable this, maybe in conjunction with a signing solution. Details TBD as it's not JDK 9. -Alan From Alan.Bateman at oracle.com Mon May 1 15:42:16 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 May 2017 16:42:16 +0100 Subject: jake -> jdk9/dev In-Reply-To: <06671bc1-3c63-4622-9a91-e8cef507a79d@default> References: <408902ef-689b-8dfa-2750-8ef629244910@oracle.com> <29b55d14-a38b-c7e4-10c2-d9259ef9a7a9@oracle.com> <06671bc1-3c63-4622-9a91-e8cef507a79d@default> Message-ID: On 01/05/2017 12:48, Stephen Felts wrote: > I think this statement is very misleading. > As part of this "small changes", jdk.attach.allowAttachSelf will be set to false and everyone that has self-attaching agents will start failing. > This is part of the revised proposal that Mark sent to the list a few weeks ago [1] -Alan [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012040.html From forax at univ-mlv.fr Mon May 1 17:37:01 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Mon, 1 May 2017 19:37:01 +0200 (CEST) Subject: Dependencies to a root modules when creating a dynamic module layer In-Reply-To: References: <1683561662.252290.1493632110367.JavaMail.zimbra@u-pem.fr> Message-ID: <1531995491.301368.1493660221141.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alan Bateman" > ?: "Remi Forax" , "jigsaw-dev" > Envoy?: Lundi 1 Mai 2017 13:41:30 > Objet: Re: Dependencies to a root modules when creating a dynamic module layer > On 01/05/2017 10:48, Remi Forax wrote: > >> Hi all, >> i try to load a kind of dynamic plugin using a ModuleLayer, >> the plugin is wrapped in a module which is loaded dynamically. >> This module has a requires directive to "java.sql" which is part of the 'root' >> modules. >> >> The problem is that my application do not requires "java.sql" because it do not >> need it, >> so "java.sql" is not part of the module loaded when the VM is booted, >> then when i load the plugin module >> - either it fails to find the required module java.sql [1] >> - or i try to load java.sql from the ModuleLayer and get an exception because >> java.sql can not be loaded by the boot classloader >> >> >> I'm sure i've missed something ? >> > There isn't general support for dynamically augmenting the platform > modules. There is partial support to allow the jdk.management.agent and > java.instrument modules be dynamically loaded but that's all at this time. In both cases, the VM adds modules to the configuration like with --add-modules before the boot layer is created, so there are not dynamically loaded. > > One thing to point out is `--add-modules ALL-DEFAULT` in JEP 261. This > is motivated by the scenario where a container needs to load > applications that require additional modules. I've decided to add a requires java.se because i know that no plugin will reference a module which is not present in java.se but like -with -add-modules ALL-DEFAULT, this is a kind of a workaround, it's pre-loading all modules because you may need one. I wonder if we can explore how to add modules to an existing configuration (if they were already accessible by the module finder used to create the configuration) in 10. > > -Alan. R?mi From alex.buckley at oracle.com Mon May 1 18:47:42 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 01 May 2017 11:47:42 -0700 Subject: Java Platform Module System In-Reply-To: <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> Message-ID: <590782CE.2090807@oracle.com> On 4/30/2017 3:25 AM, Stephan Herrmann wrote: > For the question at hand, this is what we learn from that improved > reference: > "A readability graph is constructed" > > Now we only need a link to the specification that *defines* what is a > readability graph and what is the meaning of "m1 reads m2". > I assume, you want to add a further reference to the "Resolution" > section of the package specification for java.lang.module? Yes, the API spec for java.lang.module will be updated to define the readability relation. But instead of waiting for that, I recommend you watch https://youtu.be/Vxfd3ehdAZc?t=18m15s because readability has been stable for a long time. > BTW: while pondering if the given package specification is sufficient, > I wonder if "requires transitive" should work through multiple levels: > M1 > M2 requires transitive M1 > M3 requires transitive M2 > M4 requires M3 > Does M4 read M1? It does work through multiple levels, in order to support arbitrary amounts of refactoring: once you've released a module that someone else reuses (via 'requires'), then you've committed to your module's name and API but are free to refactor its content into deeper modules which your original module "reuses" (via 'requires transitive') for the benefit of consumers. There is no "re-exporting", just modules being made to read one another. So, going top down (because resolution starts from a set of root modules) :- M4 requires and thus reads M3, and M3 requires transitive M2, so M4 reads M2. Since M4 reads M2, and M2 requires transitive M1, we have M4 reads M1. > Looking at 7.7.1: > "The requires keyword may be followed by the modifier transitive. > This causes > any module which depends on the current module to have an > implicitly declared > dependence on the module specified by the requires transitive > directive." > Am I right in assuming that "depends" should cover explicitly and > implicitly > declared dependences? Taking into consideration the subtlety about > dependence > vs. dependency, may I suggest adding s.t. like > "A module M1 is said to depend on another module M2, if it has an > explicitly > or implicitly declared dependence on M2." > (this also makes "depends" a technical term rather than the general > (fuzzy) > English word). I understand the point; when we clarify the API spec for readability, I'll make sure the JLS usage of "depends" is explicitly aligned. > Revisiting other references to "Java Platform Module System" inside JLS, > what about the two occurrences in the body of 7.7: > > - One reference is used to discriminate "dependence" from "dependency": > From a quick scan, I believe this sentence: > "Generally, the rules of the Java programming language are > more interested in dependences than dependencies." > can probably be made stronger: > "The rules of the Java programming language are not interested in > dependencies, only in dependences.". > Or perhaps the paragraph about dependencies could be removed entirely. > If this interpretation is wrong, another reference to detailed > specification > would be needed. Perhaps it is only JLS, that is agnostic to > dependencies, > whereas the API specification part indeed uses this concept? This text is an informative note distinguishing the "dependence" expressed rather statically in the Language, from the "dependency" module determined rather dynamically by the JPMS. I see no reason to change it. > - Another reference links "automatic modules" into JLS and will probably > link to ModuleFinder.of(Path...), right? This text is also an informative note. Automatic modules are discovered through ModuleFinder.of, sure, and they appear in other places in the java.lang.module API too -- but none of that is the point of the note. The point of the note is that the developer doesn't specify 'requires' any differently for an automatic module than for an explicit module. Alex From stephan.herrmann at berlin.de Mon May 1 19:23:17 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Mon, 1 May 2017 21:23:17 +0200 Subject: Java Platform Module System In-Reply-To: <5903CF7D.9060803@oracle.com> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> Message-ID: <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> On 29.04.2017 01:25, Alex Buckley wrote: > On 4/27/2017 12:38 PM, Stephan Herrmann wrote: >> Let me add a friendly reminder, that we are still waiting for a >> specification that unambiguously tells us which module system to implement. >> For illustration: >> >> (A) Is JPMS a module system that keeps the semantics of qualified names as >> they are in Java 8 and only superimposes encapsulation boundaries? >> (i.e., each type is globally uniquely identified by its qualified name). >> >> (B) Is JPMS a module system that maintains the assumption that from the >> perspective of each module all relevant types can be distinguished using >> their qualified name? >> (i.e. admitting identical qualified names as long as no compilation of one >> module will ever encounter several candidates at once). >> >> (C) Is JPMS a module system that establishes a separate namespace for each >> module, where types with identical qualified name - but defined in >> different modules - need to be distinguished? >> (i.e., uniqueness is primarily required for module-prefixed qualified >> names). >> >> Despite some efforts I fail to find a definite answer in JLS (and Alex >> mentioned that some of this is still being discussed). Still JLS as of >> today sounds mostly like (A). To me (B) sounds like the natural choice, but I >> understood Alex as saying it *should* be (C). I don't see, however, how the >> conceptual framework of JLS could possibly support such design. > > [...] > (C)-style property is now expressed in 4.3.4: > > "Two reference types are the same compile-time type if they are declared in compilation units associated with the same module > (?7.3), and they have the same binary name (?13.1), and their type arguments, if any, are the same, applying this definition > recursively." Admitted, this part looks good. And so does 6.5.5.2. Meanwhile I've come to the interpretation that the main weakness of JLS concerns handling of same-named packages in different modules. Trouble seems to start at a difference between 6.5.5.2 and 6.5.3.1/2: To identify a type, that type must be accessible from the current module, but for identifying a package, the package only needs to be visible. Ergo: identifying a package does not consider exports. Furthermore, 6.5.3.2 implies that each qualified name uniquely defines a package, when it speaks of "the member named Id within the package named by Q". Note the two occurrences of "the". This finally undermines the definition of accessibility (6.6.1), when it speaks of the "module to which the package is exported". I read this as follows: When M1 exports p1 to M2, this makes all public members of p1 accessible in M2, even those that belong to totally unrelated modules, which may not export p1. I recall Alex answering "this is still being clarified / discussed" to several questions in this area. As a result I can only conclude: JLS still doesn't tell us which module system to implement. If this were just a minor omission, why then would it still be subject to discussion, this late in the game? I see one possible explanation: changing the spec may involve much more trouble than meets the eye. Changes concerning packages are very much focusing on the hierarchical structure of packages and sub packages, despite the fact that 7.1. has always been describing this structure as having "no significance in itself". 7.4.3 already jumps through hoops, trying to balance the hierarchy-based notion of "technically observable" with the concept of "really observable" which disregards the hierarchy. In my view, a forest rooted at toplevel packages is not suitable for specifying the rules of accessibility, where each module may have a different interpretation of a given package name, in a way that is completely unrelated to hierarchy. Since "exports" refers to a package, this notion must be better aligned with modules. best, Stephan From stephan.herrmann at berlin.de Mon May 1 19:40:45 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Mon, 1 May 2017 21:40:45 +0200 Subject: Java Platform Module System In-Reply-To: <590782CE.2090807@oracle.com> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> <590782CE.2090807@oracle.com> Message-ID: On 01.05.2017 20:47, Alex Buckley wrote: > Yes, the API spec for java.lang.module will be updated to define the readability relation. thanks. > But instead of waiting for that, I > recommend you watch https://youtu.be/Vxfd3ehdAZc?t=18m15s because readability has been stable for a long time. I thought we agreed that challenging JLS is the best service we can give to the community? :) I prefer not to undermine my scrutiny with too much of an assumed intuitive understanding. >> BTW: while pondering if the given package specification is sufficient, >> I wonder if "requires transitive" should work through multiple levels: >> M1 >> M2 requires transitive M1 >> M3 requires transitive M2 >> M4 requires M3 >> Does M4 read M1? > > It does work through multiple levels, in order to support arbitrary amounts of refactoring: once you've released a module that > someone else reuses (via 'requires'), then you've committed to your module's name and API but are free to refactor its content into > deeper modules which your original module "reuses" (via 'requires transitive') for the benefit of consumers. There is no > "re-exporting", just modules being made to read one another. > > So, going top down (because resolution starts from a set of root modules) :- M4 requires and thus reads M3, and M3 requires > transitive M2, so M4 reads M2. Since M4 reads M2, and M2 requires transitive M1, we have M4 reads M1. > >> Looking at 7.7.1: >> [...] > > I understand the point; when we clarify the API spec for readability, I'll make sure the JLS usage of "depends" is explicitly aligned. Thanks, this part resolved. >> Revisiting other references to "Java Platform Module System" inside JLS, >> what about the two occurrences in the body of 7.7: >> >> - One reference is used to discriminate "dependence" from "dependency": >> From a quick scan, I believe this sentence: >> "Generally, the rules of the Java programming language are >> more interested in dependences than dependencies." >> can probably be made stronger: >> "The rules of the Java programming language are not interested in >> dependencies, only in dependences.". >> Or perhaps the paragraph about dependencies could be removed entirely. >> If this interpretation is wrong, another reference to detailed >> specification >> would be needed. Perhaps it is only JLS, that is agnostic to >> dependencies, >> whereas the API specification part indeed uses this concept? > > This text is an informative note distinguishing the "dependence" expressed rather statically in the Language, from the "dependency" > module determined rather dynamically by the JPMS. I see no reason to change it. Asked differently: when it says "Generally, the rules of the Java programming language are more interested in dependences than dependencies." which are the aspects where the rules of the Java programming language *are* interested in dependencies? >> - Another reference links "automatic modules" into JLS and will probably >> link to ModuleFinder.of(Path...), right? > > This text is also an informative note. Automatic modules are discovered through ModuleFinder.of, sure, and they appear in other > places in the java.lang.module API too -- but none of that is the point of the note. The point of the note is that the developer > doesn't specify 'requires' any differently for an automatic module than for an explicit module. You make it sound as if automatic modules are relevant only at runtime. But how then could requiring an automatic module be resolved at compile time? Asked differently: how to you plan to ensure that different compilers can resolve automatic modules in the same way? best, Stephan From Alan.Bateman at oracle.com Mon May 1 19:57:23 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 May 2017 20:57:23 +0100 Subject: Dependencies to a root modules when creating a dynamic module layer In-Reply-To: <1531995491.301368.1493660221141.JavaMail.zimbra@u-pem.fr> References: <1683561662.252290.1493632110367.JavaMail.zimbra@u-pem.fr> <1531995491.301368.1493660221141.JavaMail.zimbra@u-pem.fr> Message-ID: On 01/05/2017 18:37, forax at univ-mlv.fr wrote: > : > In both cases, the VM adds modules to the configuration like with --add-modules before the boot layer is created, so there are not dynamically loaded. That is the -Dcom.sun.management.jmxremote or -javaagent case. The other case is where a tool attaches to a running VM and starts the JMX agent or loads a java agent. If the modules aren't loaded then it trigger the creation of a relative configuration and a child layer with the modules that are needed. It's not exposed in the API so just think of it as limited support for dynamic augmentation of platform modules. It's definitely something to come back in the future. -Alan. From alex.buckley at oracle.com Mon May 1 20:06:08 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 01 May 2017 13:06:08 -0700 Subject: Java Platform Module System In-Reply-To: <1ef5ce0d-f092-b81a-5fb8-c1ea577064d2@berlin.de> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <1ef5ce0d-f092-b81a-5fb8-c1ea577064d2@berlin.de> Message-ID: <59079530.2050409@oracle.com> On 4/30/2017 4:10 AM, Stephan Herrmann wrote: > No. (B) may be true for your example, but it is not for the following > (which is similar to examples we had in our January thread): > > //-- M/module-info.java > module M { exports pm; } > > //-- M/impl/Other.java > package impl; > public class Other { } > > //-- M/pm/C1.java > package pm; > import impl.Other; > public class C1 extends Other { > public void m1(Other o) {} > } > //-- > //-- O/module-info.java > module O { requires M; } > > //-- O/impl/Other.java > package impl; > public class Other { } > > //-- O/po/Client.java > package po; > import pm.C1; > public class Client { > void test1(C1 one) { > one.m1(one); > } > } > //-- > > Looking at O, and trying to determine whether the method invocation > one.m1(one) > is legal, M's type impl.Other is *relevant*, because analysis must ... > - detect that the type reference "Other" in the signature of m1 refers > to the > type defined in M, not to the same-named type in O. > - similarly detect that the type reference in C1's class header (or > superclass > classfile attribute) refers to M's impl.Other. > - conclude from the above, that C1 is compatible to m1's parameter. > > Ergo, the set of types relevant from the perspective of O contains two > same-named types. Per 7.3, it's true that when compiling any observable ordinary compilation units associated with O (such as O/po/Client.java), the host system must limit the ordinary compilation units that would otherwise be observable, to only those that are visible to O. Since O requires M, and since M/impl/Other.java is an observable ordinary compilation unit associated with M, we have that M/impl/Other.java is visible to O. Then, by 6.3, the scope of M's top-level impl package is all observable compilation units in O. Then, we get the difficulty in 6.5.3.2, because two top-level packages called impl are visible to code in O. I specified package visibility in 7.3 in the way I did -- not considering exports -- in order to encourage compilers to take a "wide view" of what packages are physically present in required modules, even if a package isn't exported (M's package impl) and thus its types won't be accessible (M's type impl.Other isn't accessible to code in O). For example, if M's author forgets to export impl (quite possible when M is first declared), I'd like a tool processing O to have the words to say: "impl is /visible/ in M, but not exported, so none of its types -- not even its 'public' types -- can be accessed". > If Java 9 permits this situation, it not only hugely increases the > complexity > of all tools that need to "understand" this program, it also wastes a > unique > opportunity that JPMS has, which existing module systems did not have: > > Java 9 could make "API leaks" either illegal or ineffective and thus > rule out > an entire category of ill-formed programs, which to-date must unfortunately > be accepted by module-unaware compilers: > > (A) Java 9 has the opportunity to say that the declaration of m1 is > illegal, > because it publicly exposes a non-accessible type, which is broken in > every regard. > > (B) Alternatively, Java 9 has the opportunity to say that any attempt to > invoke m1 from outside M is illegal, because clients would need to know > about an inaccessible type. Understood, but we didn't take those directions. More here: https://bugs.openjdk.java.net/browse/JDK-8153362 Alex From alex.buckley at oracle.com Mon May 1 20:11:19 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 01 May 2017 13:11:19 -0700 Subject: Java Platform Module System In-Reply-To: <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> Message-ID: <59079667.2060308@oracle.com> On 5/1/2017 12:23 PM, Stephan Herrmann wrote: > Meanwhile I've come to the interpretation that the main weakness of JLS > concerns > handling of same-named packages in different modules. > > Trouble seems to start at a difference between 6.5.5.2 and 6.5.3.1/2: > To identify a type, that type must be accessible from the current module, > but for identifying a package, the package only needs to be visible. > Ergo: identifying a package does not consider exports. > > Furthermore, 6.5.3.2 implies that each qualified name uniquely defines a > package, > when it speaks of "the member named Id within the package named by Q". > Note the two occurrences of "the". Understood. I need to clarify 6.5.* to accept that multiple packages may be visible by the same name. But when we get to accessibility, only one of the visible packages should matter. > This finally undermines the definition of accessibility (6.6.1), when it > speaks > of the "module to which the package is exported". I read this as follows: > When M1 exports p1 to M2, this makes all public members of p1 accessible > in M2, > even those that belong to totally unrelated modules, which may not > export p1. > > I recall Alex answering "this is still being clarified / discussed" to > several questions in this area. > > As a result I can only conclude: JLS still doesn't tell us which module > system to implement. > > > If this were just a minor omission, why then would it still be subject to > discussion, this late in the game? I see one possible explanation: > changing the > spec may involve much more trouble than meets the eye. Changes > concerning packages > are very much focusing on the hierarchical structure of packages and sub > packages, > despite the fact that 7.1. has always been describing this structure as > having > "no significance in itself". > 7.4.3 already jumps through hoops, trying to balance the hierarchy-based > notion of > "technically observable" with the concept of "really observable" which > disregards > the hierarchy. > In my view, a forest rooted at toplevel packages is not suitable for > specifying > the rules of accessibility, where each module may have a different > interpretation > of a given package name, in a way that is completely unrelated to > hierarchy. > Since "exports" refers to a package, this notion must be better aligned > with modules. It's hard to respond to the same point in multiple sub-threads. Please see my other mail where I accept that package visibility is unrelated to 'exports'. Alex From alex.buckley at oracle.com Mon May 1 20:17:32 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 01 May 2017 13:17:32 -0700 Subject: Java Platform Module System In-Reply-To: References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> <590782CE.2090807@oracle.com> Message-ID: <590797DC.2040206@oracle.com> On 5/1/2017 12:40 PM, Stephan Herrmann wrote: > Asked differently: when it says > "Generally, the rules of the Java programming language are > more interested in dependences than dependencies." > which are the aspects where the rules of the Java programming language > *are* interested in dependencies? I'm sorry that this attempt to be helpful -- to distinguish the popular term "dependency" from the actual term "dependence" used by the JLS -- is causing so much confusion. The JLS has always contained rather open-ended text intended to add color rather than to specify an implementation. But to answer your question: When we fix up readability in the API, I expect the JLS will then speak more about the dependencies which result from resolution and are more numerous than the dependences expressed with static 'requires' directives. >>> - Another reference links "automatic modules" into JLS and will probably >>> link to ModuleFinder.of(Path...), right? >> >> This text is also an informative note. Automatic modules are >> discovered through ModuleFinder.of, sure, and they appear in other >> places in the java.lang.module API too -- but none of that is the >> point of the note. The point of the note is that the developer >> doesn't specify 'requires' any differently for an automatic module >> than for an explicit module. > > You make it sound as if automatic modules are relevant only at runtime. Huh? The JPMS is assumed to be present at compile time, not just run time. And automatic modules are a feature of the JPMS. Alex From Alan.Bateman at oracle.com Mon May 1 20:28:42 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 May 2017 21:28:42 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) Message-ID: As I mentioned in another thread, we need to get the changes accumulated in jake to jdk9/dev. JDK-8178380 [1] has the summary of the changes that have accumulated since the last refresh. One note on the hotspot repo is that the changes to dynamically augment the platform modules run into JDK-8178604 when testing with an exploded build. So there is a partial fix for this in jake. Harold has the more complete fix in review on hotspot-runtime-dev for JDK 10. The webrevs with the changes is here: http://cr.openjdk.java.net/~alanb/8178380/1/ -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8178380 From lois.foltan at oracle.com Mon May 1 20:34:08 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 1 May 2017 16:34:08 -0400 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, Hotspot changes look good. Thanks, Lois On 5/1/2017 4:28 PM, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From jonathan.gibbons at oracle.com Mon May 1 20:42:19 2017 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 01 May 2017 13:42:19 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <59079DAB.3040207@oracle.com> Langtools changes look good. -- Jon On 05/01/2017 01:28 PM, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From stephan.herrmann at berlin.de Mon May 1 20:48:07 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Mon, 1 May 2017 22:48:07 +0200 Subject: Java Platform Module System In-Reply-To: <590797DC.2040206@oracle.com> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> <590782CE.2090807@oracle.com> <590797DC.2040206@oracle.com> Message-ID: <0ac8d036-696a-54b1-2096-94d6807f21a9@berlin.de> On 01.05.2017 22:17, Alex Buckley wrote: >>>> - Another reference links "automatic modules" into JLS and will probably >>>> link to ModuleFinder.of(Path...), right? >>> >>> This text is also an informative note. Automatic modules are >>> discovered through ModuleFinder.of, sure, and they appear in other >>> places in the java.lang.module API too -- but none of that is the >>> point of the note. The point of the note is that the developer >>> doesn't specify 'requires' any differently for an automatic module >>> than for an explicit module. >> >> You make it sound as if automatic modules are relevant only at runtime. > > Huh? The JPMS is assumed to be present at compile time, not just run time. And automatic modules are a feature of the JPMS. > > Alex Wait, are you expecting compilers to actually use the implementation behind ModuleFinder etc. in order to discover modules etc.? When you said "The host system must use the Java Platform Module System (as if by execution of the 'resolve' method of java.lang.module.Configuration)" I read this as requesting a compiler to perform the same operation as is specified in that API. Was the intention behind "must use the Java Platform Module System" to say s.t. like "must call methods of API in java.lang.module"? Stephan From alex.buckley at oracle.com Mon May 1 21:06:41 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 01 May 2017 14:06:41 -0700 Subject: Java Platform Module System In-Reply-To: <0ac8d036-696a-54b1-2096-94d6807f21a9@berlin.de> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <845db0d9-4c8f-807a-467d-64b6a94cd3f9@berlin.de> <590782CE.2090807@oracle.com> <590797DC.2040206@oracle.com> <0ac8d036-696a-54b1-2096-94d6807f21a9@berlin.de> Message-ID: <5907A361.10008@oracle.com> On 5/1/2017 1:48 PM, Stephan Herrmann wrote: > On 01.05.2017 22:17, Alex Buckley wrote: >>>>> - Another reference links "automatic modules" into JLS and will >>>>> probably >>>>> link to ModuleFinder.of(Path...), right? >>>> >>>> This text is also an informative note. Automatic modules are >>>> discovered through ModuleFinder.of, sure, and they appear in other >>>> places in the java.lang.module API too -- but none of that is the >>>> point of the note. The point of the note is that the developer >>>> doesn't specify 'requires' any differently for an automatic module >>>> than for an explicit module. >>> >>> You make it sound as if automatic modules are relevant only at runtime. >> >> Huh? The JPMS is assumed to be present at compile time, not just run >> time. And automatic modules are a feature of the JPMS. >> >> Alex > > Wait, are you expecting compilers to actually use the implementation behind > ModuleFinder etc. in order to discover modules etc.? > > When you said > "The host system must use the Java Platform Module System (as if by > execution of the 'resolve' method of java.lang.module.Configuration)" > I read this as requesting a compiler to perform the same operation as is > specified in that API. > > Was the intention behind "must use the Java Platform Module System" > to say s.t. like "must call methods of API in java.lang.module"? A compiler is not required to physically invoke methods of the JPMS API. A compiler is required to determine which modules are read by each module *as if* by execution of Configuration::resolve, a Java SE API. The as-if term is common in Java SE Platform specs to mean "Implement it any way you like, but the result must agree with the result from ." Alex From kevin.rushforth at oracle.com Tue May 2 00:21:49 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Mon, 01 May 2017 17:21:49 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline Message-ID: <5907D11D.20609@oracle.com> This review is being cross-posted to both openjfx-dev and jigsaw-dev. Please review the proposed fix for: https://bugs.openjdk.java.net/browse/JDK-8177566 http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ Details of the fix as well as notes to reviewers are in the bug report [1] (e.g., I've also generated separate webrevs for the fix itself, the doc changes, and the test changes). -- Kevin [1] https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 From brian.goetz at oracle.com Tue May 2 00:38:22 2017 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 1 May 2017 20:38:22 -0400 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> Message-ID: <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Or use multi-release JARs. On 4/25/2017 3:14 AM, Jeremy Manson wrote: > I guess I can call it reflectively. From mandy.chung at oracle.com Tue May 2 02:37:08 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 May 2017 19:37:08 -0700 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke Message-ID: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> Webrev: http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8020801/webrev.00/ The big hammer check disallowing MethodHandles::lookup be called by system classes defined by the bootstrap class loader was added as defense-in-depth to prevent this caller-sensitive method being called from JDK internal classes via Method::invoke. It was intended as a point fix and to be replaced with a long-term approach. Lookup.privateLookupIn() returns a Lookup object and IAE is thrown if the lookup class is almost all java.* and sun.* [1]. We should fix this in JDK 9. This patch replaces this restriction and now allow MethodHandles::lookup to be called statically by any code. But disallow Method::invoke of MethodHandles.lookup from system classes defined by the bootstrap class loader e.g. java.base. It is expected that no reflective call to MethodHandles::lookup is made by the system classes and so this approach would provide a better mechanism as a defense-in-depth. Mandy [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012267.html From mandy.chung at oracle.com Tue May 2 02:47:04 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 May 2017 19:47:04 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> > On May 1, 2017, at 1:28 PM, Alan Bateman wrote: > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ I have reviewed all repos. Mostly looks good. This is the first round of comments. I will continue the review and plan to play a little more with the new launcher options. src/share/vm/services/attachListener.cpp 355 // Dynamic loading agents is not default by default typo: s/not default/not enabled? src/share/vm/services/diagnosticCommand.cpp 771 loadAgentModule(CHECK); 772 Handle loader = Handle(THREAD, SystemDictionary::java_system_loader()); 773 Klass* k = SystemDictionary::resolve_or_fail(vmSymbols::jdk_internal_agent_Agent(), loader, Handle(), true, CHECK); 774 instanceKlassHandle ik (THREAD, k); It?s better to refactor this to loadAgentClass that returns Klass or handle? src/java.base/share/classes/java/lang/ClassLoader.java 134 * platform class loader may delegate to application class loader. ?the? application class loader 135 * In other words, classes in modules defined to the application class 136 * loader may be visible to the platform class loader. It would help if this statement makes it clear that the classes in modules defined to the application class loader (only that are required by the upgradeable modules) are visible to the platform class loader. It would also be good to mention that ClassLoader::getPackages will not return the packages defined to the application class loader since application class loader is not its ancestor. 377 * @apiNote If the parent is specified as {@code null} (for the 378 * bootstrap class loader) then there is no guarantee that all platform 379 * classes are visible. What about: If the parent is specified as {@code null} (for the bootstrap class loader) then there is no guarantee that all platform classes are visible. Platform classes defined by the platform class loader and its ancestors except bootstrap class loader are not visible to this class loader. src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java 174 /** 175 * Register a module this this class loader. This has the effect of making 176 * the types in the module visible. 177 */ 178 public void loadModule(ModuleReference mref) { Typo in line 175 ?this this class loader?. src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java 147 // special mode to boot with only java.base, ignores other options 148 if (System.getProperty("jdk.module.minimumBoot") != null) { 149 return createMinimalBootLayer(); 150 } src/java.base/share/classes/jdk/internal/module/ModulePatcher.java 123 try (JarFile jf = new JarFile(file.toString())) { - what is the bug being fixed by this line? src/java.base/share/classes/jdk/internal/module/ModulePath.java 536 if (packages.contains(pn)) builder.mainClass(mainClass); Nit: it would be consistent if this is broken into two lines. src/java.base/share/classes/sun/launcher/LauncherHelper.java 960 bootLayer.modules().stream() 961 .map(m -> cf.findModule(m.getName())) 962 .flatMap(Optional::stream) This can be replaced with cf.modules().stream() 988 .map(e -> Stream.concat(Stream.of(e.source()), 989 toStringStream(e.modifiers())) Formatting nit: line 989 should be indented more to the right 1064 * image ot be less than modules than not in the run-time image. typo: ?ot? Should these new tracing methods drop the printToStderr argument? This argument exists for compatibility to print help message to stderr. These new tracing options will print to stdout. For validate modules, you suggested to print the errors to stderr which I agree. src/java.base/share/classes/sun/launcher/resources/launcher.properties 60 \ --limit-modules [,...]\n\ 61 \ limit the universe of observable modules\n\ -?limit-modules is intended for testing purpose. I wonder if this should be moved to ?-extra-help. 72 \ The --validate-modules option may be is useful for finding\n\ typo: ?may be is useful? src/jdk.jartool/share/classes/sun/tools/jar/Main.java 619 if (dflag) { 620 // "--describe-module/-d" does not require file argument(s), 621 // but does accept --release 622 usageError(getMsg("error.bad.dflag")); 623 return false; 624 } Mandy From mandy.chung at oracle.com Tue May 2 04:21:20 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 02 May 2017 04:21:20 +0000 Subject: hg: jigsaw/jake: Specify OS_* properties to release file at build time. Restore JDK 8 values Message-ID: <201705020421.v424LKkG016394@aojmv0008.oracle.com> Changeset: cc2f50bc047d Author: mchung Date: 2017-05-01 21:10 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/cc2f50bc047d Specify OS_* properties to release file at build time. Restore JDK 8 values ! common/autoconf/generated-configure.sh ! common/autoconf/platform.m4 ! common/autoconf/spec.gmk.in ! make/ReleaseFile.gmk From mandy.chung at oracle.com Tue May 2 04:21:43 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 02 May 2017 04:21:43 +0000 Subject: hg: jigsaw/jake/jdk: Do not define TARGET_PLATFORM property in release file Message-ID: <201705020421.v424LhM6016789@aojmv0008.oracle.com> Changeset: a20dcd573138 Author: mchung Date: 2017-05-01 21:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/a20dcd573138 Do not define TARGET_PLATFORM property in release file ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/ReleaseInfoPlugin.java ! test/tools/jlink/IntegrationTest.java From Alan.Bateman at oracle.com Tue May 2 07:48:52 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 08:48:52 +0100 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Message-ID: On 02/05/2017 07:50, Jeremy Manson wrote: > : > > If we follow this path, before we migrate to Java 9, we would need to > make sure all of our code builds and the tests pass with Java 9 and > Java 8. We can't make all of the code build and the tests pass with > Java 9 as-is, because many of them use options like Xbootclasspath, > which have been dropped. We can't migrate those tests to use the new > command line flags before we migrate to Java 9, because if we do, they > will stop working with Java 8. Due to the size of our code base, it's > pretty much impossible to migrate all of the users to a new set of > command line flags at the same time as we migrate to Java 9. So I'm curious about -Xbootclasspath as that completely overrides the runtime classes. Is this static instrumentation or combining the VM/launcher from one build with the classes from a different JDK build? In any case, and as you have found, -Xbootclasspath and -Xbootclasspath/p are meaningless now, a consequence of moving all the core APIs to modules. The -Xbootclasspath/a option works as before with the caveat that not all platform classes are visible to the boot class loader now (a consequence of ongoing security work to move non-core components away from the fully privileged boot loader). It might be that you mean the `javac -bootclasspath, this works as before when you target 8 or older. -Alan From adinn at redhat.com Tue May 2 09:29:27 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 2 May 2017 10:29:27 +0100 Subject: jake -> jdk9/dev In-Reply-To: <06671bc1-3c63-4622-9a91-e8cef507a79d@default> References: <408902ef-689b-8dfa-2750-8ef629244910@oracle.com> <29b55d14-a38b-c7e4-10c2-d9259ef9a7a9@oracle.com> <06671bc1-3c63-4622-9a91-e8cef507a79d@default> Message-ID: <70aca7cd-8e96-355c-cf1c-28a665ef36df@redhat.com> On 01/05/17 12:48, Stephen Felts wrote: > I think this statement is very misleading. As part of this "small > changes", jdk.attach.allowAttachSelf will be set to false and > everyone that has self-attaching agents will start failing. So to > repeat myself, I think that the Java community made it clear that > this feature should not be included in JDK9. Well, not exactly 'the Java community'. Quite a few people did object to the EnableDynamicAgentLoading switch being introduced and defaulting to false and as a result setting of that default was delayed until JDK10 to ease the transition (so that the switch could be added to scripts in advance of any need to move backwards and forwards between jdk9 and jdk10). Now, there was also some discussion of the agent self-hoist restriction on list but, to my recall, it was not universally addressed or discredited as you claim. Most of the discussion was concerned with how the process restrictions would apply. Anyway, at the very least, if you are going to arrogate any rights on behalf of your conception of who is 'the Java community' then, as Sam Goldwyn said, please "Include me out". I'm personally less concerned about this change of behaviour than the EnableDynamicAgentLoading change because retaining the old behaviour only requires a system property setting to be added to the command line. Since that setting will be innocuous if you move back to an older version my view is that coping with this change doesn't cause any major transition problem. I can understand the concern to avoid an /irremediable/ transition where existing, important behaviour is removed. Indeed, I can easily stretch such concern to cover an /awkward/ transition where straightforward switching between releases is prejudiced. However, to object to a change to new behaviour in circumstances where retaining the old behaviour requires a relatively innocuous change seems to me to be a much less reasonable approach. If followed repeatedly that sort of behaviour will paint Java into a corner where we can never achieve important changes in the JVM and JDK. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Tue May 2 09:40:29 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 2 May 2017 10:40:29 +0100 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> Message-ID: <06d99b2d-2140-7c22-a8a0-612de7d83b5d@redhat.com> On 25/04/17 09:19, Andrew Dinn wrote: > This discussion really ought to be happening on the Byteman forum but > anyway ... > > Yes, Alan is right that this is exactly what is going on. Byteman on > jdk9 (the 4.0.0-BETA release series) now uses method handles in place of > reflection. Unfortunately, it is limited by a slightly arbitrary > restriction on the use of method handles in java.* and sun.* packages (a > blunt instrument to provide some security guarantees). > > This is being addressed on two fronts. I am in the process of adding > unit tests to complete a workaround which remedies this problem by > falling back to reflection in the specific cases where the restriction > applies (yes, that means there is probably going to be another > 4.0.0-BETA release before jdk9 goes live %-/ ). Just wanted to post a heads-up that this fall-back behaviour has now been implemented in Byteman release 4.0.0-BETA5. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From chris.hegarty at oracle.com Tue May 2 10:11:31 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 May 2017 11:11:31 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> Message-ID: <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> Mandy, On 02/05/17 03:47, Mandy Chung wrote: > >> ... > src/jdk.jartool/share/classes/sun/tools/jar/Main.java > 619 if (dflag) { > 620 // "--describe-module/-d" does not require file argument(s), > 621 // but does accept --release > 622 usageError(getMsg("error.bad.dflag")); > 623 return false; > 624 } Sorry, I'm not sure of the issue here. I added this check when updating the describe option to accept an optional `--release` ( the effective runtime version ) when describing a multi-release modular JAR. This check is required as passing `-C`, in this context, is an error. There are exiting tests that verify this. -Chris. From peter.levart at gmail.com Tue May 2 10:14:30 2017 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 May 2017 12:14:30 +0200 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> Message-ID: <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> Hi Mandy, On 05/02/2017 04:37 AM, Mandy Chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8020801/webrev.00/ > > The big hammer check disallowing MethodHandles::lookup be called by system > classes defined by the bootstrap class loader was added as defense-in-depth > to prevent this caller-sensitive method being called from JDK internal classes > via Method::invoke. It was intended as a point fix and to be replaced > with a long-term approach. Lookup.privateLookupIn() returns a Lookup object > and IAE is thrown if the lookup class is almost all java.* and sun.* [1]. > We should fix this in JDK 9. > > This patch replaces this restriction and now allow MethodHandles::lookup to > be called statically by any code. But disallow Method::invoke of > MethodHandles.lookup from system classes defined by the bootstrap class loader > e.g. java.base. It is expected that no reflective call to > MethodHandles::lookup is made by the system classes and so this approach > would provide a better mechanism as a defense-in-depth. > > Mandy > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012267.html I don't quite understand the need for bypassing the inflation of native into generated method accessor when the reflected method is a @CallerSensitive method which has a substitute method declared to be called by reflection (like with the MethodHandles.[reflected$]lookup() pair), in the following fragment of ReflectionFactory: 179 public MethodAccessor newMethodAccessor(Method method) { 180 checkInitted(); 181 182 boolean noInflation = ReflectionFactory.noInflation; 183 if (Reflection.isCallerSensitive(method)) { 184 Method altMethod = findMethodForReflection(method); 185 if (altMethod != null) { 186 method = altMethod; 187 noInflation = true; // in this case only 188 } 189 } 190 191 if (noInflation && !ReflectUtil.isVMAnonymousClass(method.getDeclaringClass())) { 192 return new MethodAccessorGenerator(). 193 generateMethod(method.getDeclaringClass(), 194 method.getName(), 195 method.getParameterTypes(), 196 method.getReturnType(), 197 method.getExceptionTypes(), 198 method.getModifiers()); 199 } else { 200 NativeMethodAccessorImpl acc = 201 new NativeMethodAccessorImpl(method); 202 DelegatingMethodAccessorImpl res = 203 new DelegatingMethodAccessorImpl(acc); 204 acc.setParent(res); 205 return res; 206 } 207 } Is DelegatingMethodAccessorImpl/NativeMethodAccessorImpl combo not treated correctly (i.e. skipped) by the Reflection.getCallerClass(), while generated MethodAccessorImpl subclass is? Regards, Peter From alan.bateman at oracle.com Tue May 2 10:23:47 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Tue, 02 May 2017 10:23:47 +0000 Subject: hg: jigsaw/jake/jdk: 2 new changesets Message-ID: <201705021023.v42ANlk5010374@aojmv0008.oracle.com> Changeset: f8e726766d59 Author: alanb Date: 2017-05-02 07:52 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/f8e726766d59 ModuleFinder.of does not correcly handle JAR files in custom file systems ! src/java.base/share/classes/jdk/internal/module/ModulePath.java + test/java/lang/module/customfs/ModulesInCustomFileSystem.java + test/java/lang/module/customfs/m1/module-info.java + test/java/lang/module/customfs/m1/p/Main.java + test/java/lang/module/customfs/m2/module-info.java + test/java/lang/module/customfs/m2/q/Hello.java Changeset: 7234202af68f Author: alanb Date: 2017-05-02 11:22 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7234202af68f Review comments/cleanup ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java ! src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java ! src/java.base/share/classes/jdk/internal/module/ModulePath.java ! src/java.base/share/classes/sun/launcher/LauncherHelper.java ! src/java.base/share/classes/sun/launcher/resources/launcher.properties ! src/java.base/share/native/libjli/java.c From alan.bateman at oracle.com Tue May 2 10:23:58 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Tue, 02 May 2017 10:23:58 +0000 Subject: hg: jigsaw/jake/hotspot: Remove confusing comment Message-ID: <201705021023.v42ANwp5010637@aojmv0008.oracle.com> Changeset: 1fc81a6c3d44 Author: alanb Date: 2017-05-02 08:50 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/1fc81a6c3d44 Remove confusing comment ! src/share/vm/services/attachListener.cpp From adinn at redhat.com Tue May 2 10:25:19 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 2 May 2017 11:25:19 +0100 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> Message-ID: <9eb93fd8-2e40-cb0e-e1b3-cfca002f8702@redhat.com> Hi Mandy, On 02/05/17 03:37, Mandy Chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8020801/webrev.00/ > > . . . Not an official review, I am afraid, but the patch looks good and is much appreciated! regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Tue May 2 11:28:33 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 12:28:33 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> Message-ID: <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> On 02/05/2017 03:47, Mandy Chung wrote: > : > src/share/vm/services/attachListener.cpp > > 355 // Dynamic loading agents is not default by default > > typo: s/not default/not enabled? I agree, the comment is confused and better to remove it. > > src/share/vm/services/diagnosticCommand.cpp > 771 loadAgentModule(CHECK); > 772 Handle loader = Handle(THREAD, SystemDictionary::java_system_loader()); > 773 Klass* k = SystemDictionary::resolve_or_fail(vmSymbols::jdk_internal_agent_Agent(), loader, Handle(), true, CHECK); > 774 instanceKlassHandle ik (THREAD, k); > > It?s better to refactor this to loadAgentClass that returns Klass or handle? I agree although the duplicate code seems to have been there for some time, I think better if we create an issue to have it cleaned up in JDK 10. > > src/java.base/share/classes/java/lang/ClassLoader.java > > 134 * platform class loader may delegate to application class loader. > > ?the? application class loader > > 135 * In other words, classes in modules defined to the application class > 136 * loader may be visible to the platform class loader. > > It would help if this statement makes it clear that the classes in modules defined to the application class loader (only that are required by the upgradeable modules) are visible to the platform class loader. There is a typo in the javadoc, it should be "in named modules" and I think that will make it much clearer. > > It would also be good to mention that ClassLoader::getPackages will not > return the packages defined to the application class loader since > application class loader is not its ancestor. > > 377 * @apiNote If the parent is specified as {@code null} (for the > 378 * bootstrap class loader) then there is no guarantee that all platform > 379 * classes are visible. > > What about: > > If the parent is specified as {@code null} (for the bootstrap class loader) > then there is no guarantee that all platform classes are visible. > Platform classes defined by the platform class loader and its ancestors > except bootstrap class loader are not visible to this class loader. I think this would clutter the class description and it might be better if we added a note to the getPackages method. > > src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java > > 174 /** > 175 * Register a module this this class loader. This has the effect of making > 176 * the types in the module visible. > 177 */ > 178 public void loadModule(ModuleReference mref) { > > Typo in line 175 ?this this class loader?. Got it - thanks. > > src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java > > 147 // special mode to boot with only java.base, ignores other options > 148 if (System.getProperty("jdk.module.minimumBoot") != null) { > 149 return createMinimalBootLayer(); > 150 } Is there an issue here? > > src/java.base/share/classes/jdk/internal/module/ModulePatcher.java > 123 try (JarFile jf = new JarFile(file.toString())) { > - what is the bug being fixed by this line? This is part of the issue that Remi brought up last week related to overriding the default file system provider. I've replaced test/java/nio/file/spi/SetDefaultProvider.java with a more comprehensive test to test overriding with exploded and modular JAR that are patched / not-patched. > > src/java.base/share/classes/jdk/internal/module/ModulePath.java > 536 if (packages.contains(pn)) builder.mainClass(mainClass); > Nit: it would be consistent if this is broken into two lines. Okay. > > src/java.base/share/classes/sun/launcher/LauncherHelper.java > 960 bootLayer.modules().stream() > 961 .map(m -> cf.findModule(m.getName())) > 962 .flatMap(Optional::stream) > > This can be replaced with cf.modules().stream() Okay. > > 988 .map(e -> Stream.concat(Stream.of(e.source()), > 989 toStringStream(e.modifiers())) > Formatting nit: line 989 should be indented more to the right I think the IDE moved it :-) > > 1064 * image ot be less than modules than not in the run-time image. > > typo: ?ot? Okay. > > Should these new tracing methods drop the printToStderr argument? > This argument exists for compatibility to print help message to stderr. > These new tracing options will print to stdout. I was trying to keep it consistent but I think you are right, we should just init the output to System.out in these supporting methods. > For validate modules, > you suggested to print the errors to stderr which I agree. I think but it's not in jake yet. > > src/java.base/share/classes/sun/launcher/resources/launcher.properties > 60 \ --limit-modules [,...]\n\ > 61 \ limit the universe of observable modules\n\ > > -?limit-modules is intended for testing purpose. I wonder if > this should be moved to ?-extra-help. I've also been thinking it should move. > > 72 \ The --validate-modules option may be is useful for finding\n\ > > typo: ?may be is useful? Okay. Thanks for going through all the changes. I will publish a new set of webrevs tomorrow once we have all the changes. -Alan From stephan.herrmann at berlin.de Tue May 2 12:13:13 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Tue, 2 May 2017 14:13:13 +0200 Subject: Java Platform Module System In-Reply-To: <59079667.2060308@oracle.com> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> Message-ID: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> Thanks, Alex, for promising improvements in various places of the spec. I recall several threads on this list ending in you saying "still being clarified" [1][2]. Are those issues settled by now and just need to be penned down? Otherwise it would be very helpful just to see the list of open questions, so we don't bang our heads against walls that are still subject to change (I'm not speaking about the general "Issue Summary", but s.t. more focused on JLS and its dependencies in JPMS spec). Looking forward to an updated spec version, allowing us to double check if those changes raise any follow-up questions, Stephan [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-January/010866.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011544.html From Alan.Bateman at oracle.com Tue May 2 13:04:52 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 14:04:52 +0100 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5907D11D.20609@oracle.com> References: <5907D11D.20609@oracle.com> Message-ID: On 02/05/2017 01:21, Kevin Rushforth wrote: > This review is being cross-posted to both openjfx-dev and jigsaw-dev. > > Please review the proposed fix for: > > https://bugs.openjdk.java.net/browse/JDK-8177566 > http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ > > Details of the fix as well as notes to reviewers are in the bug report > [1] (e.g., I've also generated separate webrevs for the fix itself, > the doc changes, and the test changes). Overall I think the approach is okay but I wonder how easy it will be for joe developer to understand. By this I mean the description in the javadoc is lengthy and I wonder if it might be better to split up the two "solutions" so that it the choice is clearer. Related is that the IAE thrown by MethodHelper.invoke could make it clear that the package is not exported. -Alan. From jeremymanson at google.com Tue May 2 06:50:55 2017 From: jeremymanson at google.com (Jeremy Manson) Date: Mon, 1 May 2017 23:50:55 -0700 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Message-ID: Maybe when there's something breaking in Java 10 or 11, I can do that. Right now, that approach requires me to have a fork of my code, have special build goop, rely on yet another Java 9 feature, and make my relatively widely used OSS project require Java 9 to build *prior to Java 9's release -* just so I can save maybe 15-20 lines of code. Meanwhile, I'm not having a lot of luck - adding what seems to be the right reflective call doesn't seem to allow me to modify classes like ArrayList. Trying to create a boiled down repro (in 30 minute chunks between management tasks - sigh). OffTopic, but possibly of interest to you: it has also been challenging to figure out how we're going to migrate our users away from features like Xbootclasspath. Our standard operating procedure (which has worked from 5 to 6 to 7 to 8) is to get the codebase into a state where it can work both with the previous version and the new version of the JDK, and then throw the switch. If we follow this path, before we migrate to Java 9, we would need to make sure all of our code builds and the tests pass with Java 9 and Java 8. We can't make all of the code build and the tests pass with Java 9 as-is, because many of them use options like Xbootclasspath, which have been dropped. We can't migrate those tests to use the new command line flags before we migrate to Java 9, because if we do, they will stop working with Java 8. Due to the size of our code base, it's pretty much impossible to migrate all of the users to a new set of command line flags at the same time as we migrate to Java 9. Between that and the amount of reflective access we do into the JDK, we'll probably just end up re-adding features like Xbootclasspath in our fork and enabling the big kill switch for everyone, and then removing it gradually over the lifetime of Java 9. We've had to do that for some minor stuff before, but nothing this big. I'm glad we have options like that, and don't envy people with large codebases who don't. That said, we probably abuse the ability to crack open the JDK more than most people. I've been dealing with some of the concerns about upgradability for some of Google's open-source projects lately, and I am insisting that we have a rule that there is always an intermediate version that provides backwards compatibility for any desired change, so that users can always perform stepwise migrations. Thanks for the inspiration. :) Jeremy On Mon, May 1, 2017 at 5:38 PM, Brian Goetz wrote: > Or use multi-release JARs. > > > On 4/25/2017 3:14 AM, Jeremy Manson wrote: > >> I guess I can call it reflectively. >> > > From jarthana at in.ibm.com Tue May 2 14:07:12 2017 From: jarthana at in.ibm.com (Jayaprakash Arthanareeswaran) Date: Tue, 2 May 2017 14:07:12 +0000 Subject: Java Platform Module System In-Reply-To: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de>, <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> Message-ID: From jay.a at outlook.in Tue May 2 14:12:13 2017 From: jay.a at outlook.in (Jayaprakash Artanareeswaran) Date: Tue, 2 May 2017 14:12:13 +0000 Subject: Java Platform Module System Message-ID: My previous email showed up blank. So, sending from another email account. Thanks Stephan for summing it up! And at the risk of repeating myself and Stephan, we are really blocked by these issues and eagerly waiting for the updated spec. I will also use this opportunity to add one more item that should be taken to completion. This is was already discussed here but not sure there was any conclusion. Chapter 2 in the JLS [1] describes context-free grammars. The addition to "3.9 Keywords" defines "restricted keywords", which prevent the grammar for ModuleDeclaration from being context-free. This prevents compilers from using common parser generators, since those typically only support context-free grammars. The lexical/syntactic grammar split defined in chapter 2 is not of much use for actual implementations of module-info.java parsers. The spec at least needs to point out that the given grammar for ModuleDeclaration is not actually context-free. Regards, Jay [1] http://cr.openjdk.java.net/~mr/jigsaw/spec/java-se-9-jls-pr-diffs.pdf (2017-02-22) ----- Original message ----- From: Stephan Herrmann Sent by: "jigsaw-dev" To: Alex Buckley , jigsaw-dev at openjdk.java.net Cc: Subject: Re: Java Platform Module System Date: Tue, May 2, 2017 5:43 PM Thanks, Alex, for promising improvements in various places of the spec. I recall several threads on this list ending in you saying "still being clarified" [1][2]. Are those issues settled by now and just need to be penned down? Otherwise it would be very helpful just to see the list of open questions, so we don't bang our heads against walls that are still subject to change (I'm not speaking about the general "Issue Summary", but s.t. more focused on JLS and its dependencies in JPMS spec). Looking forward to an updated spec version, allowing us to double check if those changes raise any follow-up questions, Stephan [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-January/010866.html [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011544.html From brian.goetz at oracle.com Tue May 2 14:51:19 2017 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 2 May 2017 10:51:19 -0400 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Message-ID: <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> 8 can deal with multi-release JARs. The extra goop is ignored. So a 7/8/9 MRJar is perfectly possible (and in fact, the whole point of doing them in the first place, so library writers could publish JARs that span 7/8/9.) On 5/2/2017 2:50 AM, Jeremy Manson wrote: > Maybe when there's something breaking in Java 10 or 11, I can do > that. Right now, that approach requires me to have a fork of my code, > have special build goop, rely on yet another Java 9 feature, and make > my relatively widely used OSS project require Java 9 to build /prior > to Java 9's release -/ just so I can save maybe 15-20 lines of code. > > Meanwhile, I'm not having a lot of luck - adding what seems to be the > right reflective call doesn't seem to allow me to modify classes like > ArrayList. Trying to create a boiled down repro (in 30 minute chunks > between management tasks - sigh). > > OffTopic, but possibly of interest to you: it has also been > challenging to figure out how we're going to migrate our users away > from features like Xbootclasspath. Our standard operating procedure > (which has worked from 5 to 6 to 7 to 8) is to get the codebase into a > state where it can work both with the previous version and the new > version of the JDK, and then throw the switch. > > If we follow this path, before we migrate to Java 9, we would need to > make sure all of our code builds and the tests pass with Java 9 and > Java 8. We can't make all of the code build and the tests pass with > Java 9 as-is, because many of them use options like Xbootclasspath, > which have been dropped. We can't migrate those tests to use the new > command line flags before we migrate to Java 9, because if we do, they > will stop working with Java 8. Due to the size of our code base, it's > pretty much impossible to migrate all of the users to a new set of > command line flags at the same time as we migrate to Java 9. > > Between that and the amount of reflective access we do into the JDK, > we'll probably just end up re-adding features like Xbootclasspath in > our fork and enabling the big kill switch for everyone, and then > removing it gradually over the lifetime of Java 9. We've had to do > that for some minor stuff before, but nothing this big. I'm glad we > have options like that, and don't envy people with large codebases who > don't. > > That said, we probably abuse the ability to crack open the JDK more > than most people. > > I've been dealing with some of the concerns about upgradability for > some of Google's open-source projects lately, and I am insisting that > we have a rule that there is always an intermediate version that > provides backwards compatibility for any desired change, so that users > can always perform stepwise migrations. Thanks for the inspiration. :) > > Jeremy > > On Mon, May 1, 2017 at 5:38 PM, Brian Goetz > wrote: > > Or use multi-release JARs. > > > On 4/25/2017 3:14 AM, Jeremy Manson wrote: > > I guess I can call it reflectively. > > > From kevin.rushforth at oracle.com Tue May 2 15:09:39 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 02 May 2017 08:09:39 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> Message-ID: <5908A133.4020808@oracle.com> I presume you are talking about the short description I added to the beans classes and to the select binding methods (and, for that matter, the *PropertyValueFactory classes)? I agree that it does seem a bit terse, so you are right that developers may have a hard time understanding it. Without duplicating the "deploying an application as a module" section from the Introduction to FXML -- which would seem out of place in a JavaBeansXXXX class description or a selectXXXX() method description -- can you think of a good way to make it more clear? Btw, the following is the IAE exception that will be thrown: java.lang.IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base This is basically the same IAE message that would happen if we were to invoke the method directly. -- Kevin Alan Bateman wrote: > On 02/05/2017 01:21, Kevin Rushforth wrote: > >> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >> >> Please review the proposed fix for: >> >> https://bugs.openjdk.java.net/browse/JDK-8177566 >> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >> >> Details of the fix as well as notes to reviewers are in the bug >> report [1] (e.g., I've also generated separate webrevs for the fix >> itself, the doc changes, and the test changes). > Overall I think the approach is okay but I wonder how easy it will be > for joe developer to understand. By this I mean the description in the > javadoc is lengthy and I wonder if it might be better to split up the > two "solutions" so that it the choice is clearer. Related is that the > IAE thrown by MethodHelper.invoke could make it clear that the package > is not exported. > > -Alan. From mandy.chung at oracle.com Tue May 2 15:37:53 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 08:37:53 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> <49754fcf-0016-424e-e5d7-5ae59ab3b7a8@oracle.com> Message-ID: > On May 2, 2017, at 4:28 AM, Alan Bateman wrote: > > I think this would clutter the class description and it might be better if we added a note to the getPackages method. > That?s an alternative. >> src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java >> >> 147 // special mode to boot with only java.base, ignores other options >> 148 if (System.getProperty("jdk.module.minimumBoot") != null) { >> 149 return createMinimalBootLayer(); >> 150 } > Is there an issue here? > Not an issue. My mistake forgetting to go back to take this out after went through the webrevs. > >> >> src/java.base/share/classes/jdk/internal/module/ModulePatcher.java >> 123 try (JarFile jf = new JarFile(file.toString())) { >> - what is the bug being fixed by this line? > This is part of the issue that Remi brought up last week related to overriding the default file system provider. I've replaced test/java/nio/file/spi/SetDefaultProvider.java with a more comprehensive test to test overriding with exploded and modular JAR that are patched / not-patched. > Thanks. I now remember it. Mandy From mandy.chung at oracle.com Tue May 2 15:40:09 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 08:40:09 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> References: <072A3264-1459-4E76-B7B7-11FD4C838602@oracle.com> <44af0927-5fa5-7106-1a35-17a93c80e91d@oracle.com> Message-ID: <42973DF2-E3CB-486C-AC15-F16B43F7C594@oracle.com> > On May 2, 2017, at 3:11 AM, Chris Hegarty wrote: > > Mandy, > > On 02/05/17 03:47, Mandy Chung wrote: >> >>> ... >> src/jdk.jartool/share/classes/sun/tools/jar/Main.java >> 619 if (dflag) { >> 620 // "--describe-module/-d" does not require file argument(s), >> 621 // but does accept --release >> 622 usageError(getMsg("error.bad.dflag")); >> 623 return false; >> 624 } > > Sorry, I'm not sure of the issue here. I added this check when > updating the describe option to accept an optional `--release` > ( the effective runtime version ) when describing a multi-release > modular JAR. This check is required as passing `-C`, in this > context, is an error. There are exiting tests that verify this. Not an issue. I was a little confused of this change at the beginning mostly due to the jar tool implementation. This is fine. Mandy From David.Hill at Oracle.com Tue May 2 15:52:48 2017 From: David.Hill at Oracle.com (David Hill) Date: Tue, 02 May 2017 11:52:48 -0400 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> Message-ID: <5908AB50.8080504@Oracle.com> Current: All classes and properties used in a select-binding have to be declared public. Additionally, if any class is in a named module, then the module must |open| the containing package for that class to at least the |javafx.base| module (or |export| the containing package unconditionally). Suggestion: All classes and properties used in a select-binding have to be declared public. If any class is in a named module, then the module or package containing the class needs to be open or include |javafx.base in the export list.| any better ? On 5/2/17, 9:04 AM, Alan Bateman wrote: > On 02/05/2017 01:21, Kevin Rushforth wrote: > >> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >> >> Please review the proposed fix for: >> >> https://bugs.openjdk.java.net/browse/JDK-8177566 >> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >> >> Details of the fix as well as notes to reviewers are in the bug report [1] (e.g., I've also generated separate webrevs for the fix itself, the doc changes, and the test changes). > Overall I think the approach is okay but I wonder how easy it will be for joe developer to understand. By this I mean the description in the javadoc is lengthy and I wonder if it might be better to split up the two "solutions" so that it the choice is clearer. Related is that the IAE thrown by MethodHelper.invoke could make it clear that the package is not exported. > > -Alan. -- David Hill Java Embedded Development "A man's feet should be planted in his country, but his eyes should survey the world." -- George Santayana (1863 - 1952) From kevin.rushforth at oracle.com Tue May 2 16:04:07 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 02 May 2017 09:04:07 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5908AB50.8080504@Oracle.com> References: <5907D11D.20609@oracle.com> <5908AB50.8080504@Oracle.com> Message-ID: <5908ADF7.5020803@oracle.com> David Hill wrote: > > Current: > All classes and properties used in a select-binding have to be > declared public. Additionally, if any class is in a named module, then > the module must |open| > > the containing package for that class to at least the |javafx.base| > module (or |export| > > the containing package unconditionally). > > Suggestion: > > All classes and properties used in a select-binding have to be > declared public. If any class is in a named module, then the module or > package containing the class needs to be open or include |javafx.base > in the export list.| > > any better ? > It might read better, but is now incorrect. It is not sufficient for a module to *export* the package to javafx.base. Nor is it required that the module be *open* unconditionally. Rather it must *open* the package to javafx.base. The alternative is to export it unconditionally as public API of the app: WORKS: module foo.app { opens com.foo to javafx.base; } WORKS: module foo.app { exports com.foo; } WORKS (but overkill): module foo.app { opens com.foo; } FAILS: module foo.app { exports com.foo to javafx.base; } FAILS: module foo.app { /* does not export or open com.foo */ } -- Kevin > > > On 5/2/17, 9:04 AM, Alan Bateman wrote: >> On 02/05/2017 01:21, Kevin Rushforth wrote: >> >>> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >>> >>> Please review the proposed fix for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8177566 >>> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >>> >>> Details of the fix as well as notes to reviewers are in the bug >>> report [1] (e.g., I've also generated separate webrevs for the fix >>> itself, the doc changes, and the test changes). >> Overall I think the approach is okay but I wonder how easy it will be >> for joe developer to understand. By this I mean the description in >> the javadoc is lengthy and I wonder if it might be better to split up >> the two "solutions" so that it the choice is clearer. Related is that >> the IAE thrown by MethodHelper.invoke could make it clear that the >> package is not exported. >> >> -Alan. > > > -- > David Hill > Java Embedded Development > > "A man's feet should be planted in his country, but his eyes should survey the world." > -- George Santayana (1863 - 1952) > From misterm at gmail.com Tue May 2 16:45:12 2017 From: misterm at gmail.com (Michael Nascimento) Date: Tue, 2 May 2017 13:45:12 -0300 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> Message-ID: On Tue, May 2, 2017 at 11:51 AM, Brian Goetz wrote: > 8 can deal with multi-release JARs. > I missed that. Starting with what update? Is there any link I could use to share this info? Regards, Michael From jeremymanson at google.com Tue May 2 16:46:14 2017 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 2 May 2017 09:46:14 -0700 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> Message-ID: Brian: Bit of a miscommunication on my part: I know 8 can deal with multirelease JARs. I just can't build them with 8 (although maybe I could with enough build magic). When I said "rely on another 9 feature", I just meant that I would be building something else in that required 9, which would be bad because I would require a pre-release 9 to build. I'm not going to inflict that on my users (many of whom have forked it and build it themselves). (This project isn't Guava or Dagger or something - it's just a small-ball instrumentation agent.) Alan: Most of our code runs with our home-grown launcher (so we can take advantage of JEP 178) and an OpenJDK-based JVM, although we also still build and distribute the OpenJDK launcher, and many people use it. People are using Xbootclasspath for a variety of things. We use it in various non-OpenJDK libraries configurations (like with GWT and j2objc) to run with their class libraries, which are not OpenJDK-based. We also have folks using Jetty ALPN, which requires it. We also have a bunch of people using it for instrumentation agents (it's a little tricky to get X-Boot-Class-Path to work in our environment), but there should be workarounds for that. We have a few people using it for... not-immediately-obvious reasons. I didn't realize that bootclasspath/a still partially worked - that might be enough for some of these use cases. We'll have to see when we get to it. And see what happens when the stuff hits the fan. We don't actually use javac at all - we have our own wrapper around the javac API. Anyway, I can see we have an interesting upgrade path ahead of us. I'm glad that we solved so many recurring problems permanently (like our hash randomization change). Jeremy On Tue, May 2, 2017 at 7:51 AM, Brian Goetz wrote: > 8 can deal with multi-release JARs. The extra goop is ignored. So a > 7/8/9 MRJar is perfectly possible (and in fact, the whole point of doing > them in the first place, so library writers could publish JARs that span > 7/8/9.) > > > On 5/2/2017 2:50 AM, Jeremy Manson wrote: > > Maybe when there's something breaking in Java 10 or 11, I can do that. > Right now, that approach requires me to have a fork of my code, have > special build goop, rely on yet another Java 9 feature, and make my > relatively widely used OSS project require Java 9 to build *prior to Java > 9's release -* just so I can save maybe 15-20 lines of code. > > Meanwhile, I'm not having a lot of luck - adding what seems to be the > right reflective call doesn't seem to allow me to modify classes like > ArrayList. Trying to create a boiled down repro (in 30 minute chunks > between management tasks - sigh). > > OffTopic, but possibly of interest to you: it has also been challenging to > figure out how we're going to migrate our users away from features like > Xbootclasspath. Our standard operating procedure (which has worked from 5 > to 6 to 7 to 8) is to get the codebase into a state where it can work both > with the previous version and the new version of the JDK, and then throw > the switch. > > If we follow this path, before we migrate to Java 9, we would need to make > sure all of our code builds and the tests pass with Java 9 and Java 8. We > can't make all of the code build and the tests pass with Java 9 as-is, > because many of them use options like Xbootclasspath, which have been > dropped. We can't migrate those tests to use the new command line flags > before we migrate to Java 9, because if we do, they will stop working with > Java 8. Due to the size of our code base, it's pretty much impossible to > migrate all of the users to a new set of command line flags at the same > time as we migrate to Java 9. > > Between that and the amount of reflective access we do into the JDK, we'll > probably just end up re-adding features like Xbootclasspath in our fork and > enabling the big kill switch for everyone, and then removing it gradually > over the lifetime of Java 9. We've had to do that for some minor stuff > before, but nothing this big. I'm glad we have options like that, and > don't envy people with large codebases who don't. > > That said, we probably abuse the ability to crack open the JDK more than > most people. > > I've been dealing with some of the concerns about upgradability for some > of Google's open-source projects lately, and I am insisting that we have a > rule that there is always an intermediate version that provides backwards > compatibility for any desired change, so that users can always perform > stepwise migrations. Thanks for the inspiration. :) > > Jeremy > > On Mon, May 1, 2017 at 5:38 PM, Brian Goetz > wrote: > >> Or use multi-release JARs. >> >> >> On 4/25/2017 3:14 AM, Jeremy Manson wrote: >> >>> I guess I can call it reflectively. >>> >> >> > > From mandy.chung at oracle.com Tue May 2 16:56:49 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 09:56:49 -0700 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> Message-ID: > On May 2, 2017, at 3:14 AM, Peter Levart wrote: > > I don't quite understand the need for bypassing the inflation of native into generated method accessor The VM native reflection implementation does not know about this alternate `reflected$XXX` mechanism. No VM change in this patch and so we ensure this be called via the generated bytecode for Method::invoke rather than going through the VM native reflection which reduces startup overhead. > Is DelegatingMethodAccessorImpl/NativeMethodAccessorImpl combo not treated correctly (i.e. skipped) by the Reflection.getCallerClass(), while generated MethodAccessorImpl subclass is? As this case is forced not to go through VM reflection support, unless I miss something, DelegatingMethodAccessorImpl/NativeMethodAccessorImpl has no need to be changed. I will double check with the VM runtime team. Mandy From paul.sandoz at oracle.com Tue May 2 17:15:24 2017 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 May 2017 10:15:24 -0700 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> Message-ID: <1D3520A6-FFCD-4253-ABD6-CE9E7B7BFA44@oracle.com> > On 2 May 2017, at 09:45, Michael Nascimento wrote: > > On Tue, May 2, 2017 at 11:51 AM, Brian Goetz wrote: > >> 8 can deal with multi-release JARs. >> > > I missed that. Starting with what update? Is there any link I could use to > share this info? > There have been no modifications to the Java 8 runtime, it will just ignore stuff in the META-INF/versions/{N} directories. At one point we were mulling back porting the Java 9 runtime changes to Java 8 and therefore we could go back to Java 7, but we decided not to proceed with that, which is what i suspect you may be thinking about. Paul. From misterm at gmail.com Tue May 2 17:18:06 2017 From: misterm at gmail.com (Michael Nascimento) Date: Tue, 2 May 2017 14:18:06 -0300 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <1D3520A6-FFCD-4253-ABD6-CE9E7B7BFA44@oracle.com> References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> <3aef86fc-3fc2-9ff9-7be4-06d92800f0a7@oracle.com> <1D3520A6-FFCD-4253-ABD6-CE9E7B7BFA44@oracle.com> Message-ID: On Tue, May 2, 2017 at 2:15 PM, Paul Sandoz wrote: > At one point we were mulling back porting the Java 9 runtime changes to > Java 8 and therefore we could go back to Java 7, but we decided not to > proceed with that, which is what i suspect you may be thinking about. > > Yes, I was hopeful about the last bit. Thank you for the clarification! Regards, Michael From forax at univ-mlv.fr Tue May 2 18:00:05 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 2 May 2017 20:00:05 +0200 (CEST) Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Message-ID: <1143221410.1116799.1493748005687.JavaMail.zimbra@u-pem.fr> Hi Kirk, --patch-module both at compile time and at runtime is what your are looking for. cheers, R?mi ----- Mail original ----- > De: "Kirk Pepperdine" > ?: "Alan Bateman" > Cc: "jigsaw-dev" , core-libs-dev at openjdk.java.net > Envoy?: Mardi 2 Mai 2017 19:50:50 > Objet: Re: Accessing module internals from bytecode rewriting agent > Hi Alan, > > One of the techniques that I?ve used in the past for troubleshooting is to > prepend modified classes to the bootstrap class path. I fear all this mucking > about with modularization is going to leave the road to diagnostics littered > with a ton of tools that will not longer work. As it is, 8 broke all the > techniques and tooling used to diagnose class loader leaks? without fixing the > fundamental problems that caused them to leak in the first place..and added a > new sources for performance regressions for which there is little in the way of > tooling and certainly much less of an understanding of the fundamental issues > out in the general developer population. The availability of high quality > tooling is a huge bonus in the Java eco structure?. Please, don?t send us back > to the Stone Age. > > Kind regards, > Kirk Pepperdine > >> On May 2, 2017, at 9:48 AM, Alan Bateman wrote: >> >> On 02/05/2017 07:50, Jeremy Manson wrote: >>> : >>> >>> If we follow this path, before we migrate to Java 9, we would need to make sure >>> all of our code builds and the tests pass with Java 9 and Java 8. We can't >>> make all of the code build and the tests pass with Java 9 as-is, because many >>> of them use options like Xbootclasspath, which have been dropped. We can't >>> migrate those tests to use the new command line flags before we migrate to Java >>> 9, because if we do, they will stop working with Java 8. Due to the size of >>> our code base, it's pretty much impossible to migrate all of the users to a new >>> set of command line flags at the same time as we migrate to Java 9. >> So I'm curious about -Xbootclasspath as that completely overrides the runtime >> classes. Is this static instrumentation or combining the VM/launcher from one >> build with the classes from a different JDK build? In any case, and as you have >> found, -Xbootclasspath and -Xbootclasspath/p are meaningless now, a >> consequence of moving all the core APIs to modules. The -Xbootclasspath/a >> option works as before with the caveat that not all platform classes are >> visible to the boot class loader now (a consequence of ongoing security work to >> move non-core components away from the fully privileged boot loader). >> >> It might be that you mean the `javac -bootclasspath, this works as before when >> you target 8 or older. >> >> -Alan From stanislav.lukyanov at oracle.com Tue May 2 19:21:25 2017 From: stanislav.lukyanov at oracle.com (stanislav lukyanov) Date: Tue, 2 May 2017 22:21:25 +0300 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, In the Launcher-Agent-Class, would it make sense to use 'premain' instead of 'agentmain' to align it with Premain-Class rather than with Agent-Class? (maybe even rename Launcher-Agent-Class to Launcher-Premain-Class?) AFAIU Launcher-Agent-Class is meant to replace a particular use case of self-attach in applications - hence the 'agentmain'. But on the other hand, it is essentially a "shortcut" for '-jar app.jar -javaagent:app.jar', so 'premain' might as well be expected to be used (especially by a new user). Also Launcher-Agent-Class is invoked before the application, like Premain-Class and unlike Agent-Class, and some agents out there might use the entry method (premain or agentmain declared by the same class) to determine how they were invoked. It will not be possible if 'agentmain' can be invoked both before and after 'main' (using different classes for Premain-Class, Agent-Class and Launcher-Agent-Class will still work though). Thanks, Stas On 01.05.2017 23:28, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From kevin.rushforth at oracle.com Tue May 2 19:44:24 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 02 May 2017 12:44:24 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5908ADF7.5020803@oracle.com> References: <5907D11D.20609@oracle.com> <5908AB50.8080504@Oracle.com> <5908ADF7.5020803@oracle.com> Message-ID: <5908E198.302@oracle.com> Is this any better? It adds an example module-info.java file to JavaBeanXXX as is done in the FXML class. ------------------------------------------------------------------------------------------ NOTE: If the Java Bean class is in a named module, then it must be reflectively accessible to the javafx.base module. A class is reflectively accessible if the module "opens" the containing package to at least the javafx.base module. For example, if com.foo.MyBeanClass is in the foo.app module, the module-info.java might look like this: module foo.app { opens com.foo to javafx.base; } Alternatively, a class is also reflectively accessible if the module "exports" the containing package unconditionally. ------------------------------------------------------------------------------------------ Comments? -- Kevin Kevin Rushforth wrote: > > > David Hill wrote: >> >> Current: >> All classes and properties used in a select-binding have to be >> declared public. Additionally, if any class is in a named module, >> then the module must |open| >> >> the containing package for that class to at least the |javafx.base| >> module (or |export| >> >> the containing package unconditionally). >> >> Suggestion: >> >> All classes and properties used in a select-binding have to be >> declared public. If any class is in a named module, then the module >> or package containing the class needs to be open or include >> |javafx.base in the export list.| >> >> any better ? >> > > It might read better, but is now incorrect. It is not sufficient for a > module to *export* the package to javafx.base. Nor is it required that > the module be *open* unconditionally. Rather it must *open* the > package to javafx.base. The alternative is to export it > unconditionally as public API of the app: > > WORKS: module foo.app { opens com.foo to javafx.base; } > WORKS: module foo.app { exports com.foo; } > WORKS (but overkill): module foo.app { opens com.foo; } > > FAILS: module foo.app { exports com.foo to javafx.base; } > FAILS: module foo.app { /* does not export or open com.foo */ } > > > -- Kevin > > > > > > >> >> >> On 5/2/17, 9:04 AM, Alan Bateman wrote: >>> On 02/05/2017 01:21, Kevin Rushforth wrote: >>> >>>> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >>>> >>>> Please review the proposed fix for: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8177566 >>>> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >>>> >>>> Details of the fix as well as notes to reviewers are in the bug >>>> report [1] (e.g., I've also generated separate webrevs for the fix >>>> itself, the doc changes, and the test changes). >>> Overall I think the approach is okay but I wonder how easy it will >>> be for joe developer to understand. By this I mean the description >>> in the javadoc is lengthy and I wonder if it might be better to >>> split up the two "solutions" so that it the choice is clearer. >>> Related is that the IAE thrown by MethodHelper.invoke could make it >>> clear that the package is not exported. >>> >>> -Alan. >> >> >> -- >> David Hill >> Java Embedded Development >> >> "A man's feet should be planted in his country, but his eyes should >> survey the world." >> -- George Santayana (1863 - 1952) >> From peter.levart at gmail.com Tue May 2 20:17:28 2017 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 May 2017 22:17:28 +0200 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> Message-ID: <08f27f02-2585-9544-7b70-fb55d9062dc2@gmail.com> On 05/02/2017 06:56 PM, Mandy Chung wrote: >> On May 2, 2017, at 3:14 AM, Peter Levart wrote: >> >> I don't quite understand the need for bypassing the inflation of native into generated method accessor > The VM native reflection implementation does not know about this alternate `reflected$XXX` mechanism. No VM change in this patch and so we ensure this be called via the generated bytecode for Method::invoke rather than going through the VM native reflection which reduces startup overhead. I don't think VM native reflection implementation needs to know anything about this alternate `reflected$XXX` mechanism. The NativeMethodAccessorImpl is constructed with the Method argument. In case of `reflected$XXX` mechanism, it is given the alternate Method object that points to the alternate method, so NativeMethodAccessorImpl.invoke0 is called with this alternate Method object. It is like reflecting over the alternate method itself, isn't it? Am I missing something? >> Is DelegatingMethodAccessorImpl/NativeMethodAccessorImpl combo not treated correctly (i.e. skipped) by the Reflection.getCallerClass(), while generated MethodAccessorImpl subclass is? > As this case is forced not to go through VM reflection support, unless I miss something, DelegatingMethodAccessorImpl/NativeMethodAccessorImpl has no need to be changed. I will double check with the VM runtime team. I was asking because I suspected that this might be the reason for skipping DelegatingMethodAccessorImpl/NativeMethodAccessorImpl. But if it is not the reason (and anyway it would be a bug because other @CallerSensitive methods would behave erratically if this was the case), then I still don't see a reason for skipping DelegatingMethodAccessorImpl/NativeMethodAccessorImpl and proceeding directly with generated method accessor. Regards, Peter > Mandy > From Alan.Bateman at oracle.com Tue May 2 20:45:14 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 21:45:14 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <373dbc2e-5993-e2a2-f3ac-fa96b195d53c@oracle.com> On 02/05/2017 20:21, stanislav lukyanov wrote: > Hi Alan, > > In the Launcher-Agent-Class, would it make sense to use 'premain' > instead of 'agentmain' > to align it with Premain-Class rather than with Agent-Class? > (maybe even rename Launcher-Agent-Class to Launcher-Premain-Class?) > > AFAIU Launcher-Agent-Class is meant to replace a particular use case > of self-attach in applications - hence the 'agentmain'. > But on the other hand, it is essentially a "shortcut" for '-jar > app.jar -javaagent:app.jar', so 'premain' might as well be expected to > be used (especially by a new user). > > Also Launcher-Agent-Class is invoked before the application, like > Premain-Class and unlike Agent-Class, > and some agents out there might use the entry method (premain or > agentmain declared by the same class) to determine how they were invoked. > It will not be possible if 'agentmain' can be invoked both before and > after 'main' > (using different classes for Premain-Class, Agent-Class and > Launcher-Agent-Class will still work though). Executable JARs is a launcher concept, specified in the JAR file spec, and not something that the VM knows anything about. You won't see anything about executable JARs in the JNI invocation interface for example. This means that the agent declared via Launcher-Agent-Class is started in a running VM and is more like Agent-Class/agentmain than Premain-Class/premain. In particular, it may not be possible to get all the capabilities that an agent specified via `-javaagent` can get. So I think the proposed names are okay. As regards using an executable JAR with `-javaagent`, or a tool loading the executable JAR as an agent library into a running VM, then I think there is enough flexibility with using different class names. -Alan. From Alan.Bateman at oracle.com Tue May 2 20:46:40 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 May 2017 21:46:40 +0100 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <1143221410.1116799.1493748005687.JavaMail.zimbra@u-pem.fr> References: <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> <1143221410.1116799.1493748005687.JavaMail.zimbra@u-pem.fr> Message-ID: <4fde39fa-bc73-5400-8e81-b6508c7447d4@oracle.com> On 02/05/2017 19:00, Remi Forax wrote: > Hi Kirk, > --patch-module both at compile time and at runtime is what your are looking for. > > Right, and more details in the "Patching module content" section of JEP 261 [1]. -Alan [1] http://openjdk.java.net/jeps/261#Patching-module-content From mandy.chung at oracle.com Tue May 2 20:59:13 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 13:59:13 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5907D11D.20609@oracle.com> References: <5907D11D.20609@oracle.com> Message-ID: Hi Kevin, > On May 1, 2017, at 5:21 PM, Kevin Rushforth wrote: > > This review is being cross-posted to both openjfx-dev and jigsaw-dev. > > Please review the proposed fix for: > > https://bugs.openjdk.java.net/browse/JDK-8177566 > http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ First pass of comment: javafx.base/src/main/java/com/sun/javafx/property/PropertyReference.java 196 try { 197 return (ReadOnlyProperty)MethodHelper.invoke(propertyGetter, bean, (Object[])null); 198 } catch (Exception ex) { 199 throw new RuntimeException(ex); 200 } Do you have an example exception thrown if the package is not open to javafx.base? IAE is thrown by MethodHelper.invoke. Are you detecting this and throw an exception with friendlier message? javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java javafx.web/src/main/java/com/sun/webkit/MethodHelper.java 45 public static Object invoke(Method m, Object obj, Object[] params) To avoid 3 ModuleHelper classes, the invoke method can take the callerModule argument to replace this line: 56 final Module thisModule = MethodHelper.class.getModule(); javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java There are a few other public methods which I think JavaFX doesn?t need and can be removed. Mandy From mandy.chung at oracle.com Tue May 2 21:15:27 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Tue, 02 May 2017 21:15:27 +0000 Subject: hg: jigsaw/jake/jdk: Remove JAVA_FULL_VERSION from release file. Not present in JDK 8 Message-ID: <201705022115.v42LFRh2005828@aojmv0008.oracle.com> Changeset: 79750f9d95d5 Author: mchung Date: 2017-05-02 14:14 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/79750f9d95d5 Remove JAVA_FULL_VERSION from release file. Not present in JDK 8 ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/ReleaseInfoPlugin.java From kirk.pepperdine at gmail.com Tue May 2 17:50:50 2017 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 2 May 2017 19:50:50 +0200 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <64a587a9-ec06-f0cd-f136-149b85e083d7@oracle.com> Message-ID: Hi Alan, One of the techniques that I?ve used in the past for troubleshooting is to prepend modified classes to the bootstrap class path. I fear all this mucking about with modularization is going to leave the road to diagnostics littered with a ton of tools that will not longer work. As it is, 8 broke all the techniques and tooling used to diagnose class loader leaks? without fixing the fundamental problems that caused them to leak in the first place..and added a new sources for performance regressions for which there is little in the way of tooling and certainly much less of an understanding of the fundamental issues out in the general developer population. The availability of high quality tooling is a huge bonus in the Java eco structure?. Please, don?t send us back to the Stone Age. Kind regards, Kirk Pepperdine > On May 2, 2017, at 9:48 AM, Alan Bateman wrote: > > On 02/05/2017 07:50, Jeremy Manson wrote: >> : >> >> If we follow this path, before we migrate to Java 9, we would need to make sure all of our code builds and the tests pass with Java 9 and Java 8. We can't make all of the code build and the tests pass with Java 9 as-is, because many of them use options like Xbootclasspath, which have been dropped. We can't migrate those tests to use the new command line flags before we migrate to Java 9, because if we do, they will stop working with Java 8. Due to the size of our code base, it's pretty much impossible to migrate all of the users to a new set of command line flags at the same time as we migrate to Java 9. > So I'm curious about -Xbootclasspath as that completely overrides the runtime classes. Is this static instrumentation or combining the VM/launcher from one build with the classes from a different JDK build? In any case, and as you have found, -Xbootclasspath and -Xbootclasspath/p are meaningless now, a consequence of moving all the core APIs to modules. The -Xbootclasspath/a option works as before with the caveat that not all platform classes are visible to the boot class loader now (a consequence of ongoing security work to move non-core components away from the fully privileged boot loader). > > It might be that you mean the `javac -bootclasspath, this works as before when you target 8 or older. > > -Alan > From kevin.rushforth at oracle.com Tue May 2 21:22:49 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 02 May 2017 14:22:49 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> Message-ID: <5908F8A9.1000205@oracle.com> inline Mandy Chung wrote: > Hi Kevin, > > >> On May 1, 2017, at 5:21 PM, Kevin Rushforth wrote: >> >> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >> >> Please review the proposed fix for: >> >> https://bugs.openjdk.java.net/browse/JDK-8177566 >> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >> > > First pass of comment: > > javafx.base/src/main/java/com/sun/javafx/property/PropertyReference.java > 196 try { > 197 return (ReadOnlyProperty)MethodHelper.invoke(propertyGetter, bean, (Object[])null); > > 198 } catch (Exception ex) { > 199 throw new RuntimeException(ex); > 200 } > > Do you have an example exception thrown if the package is not > open to javafx.base? IAE is thrown by MethodHelper.invoke. > Are you detecting this and throw an exception with friendlier > message? > Here is the message: IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base It is roughly the same message that any similar illegal access would generate (e.g., we get similar error messages when FXML tries to call setAccessible for a field annotated with @FXML if the module is not "open" to javafx.fxml). > javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java > javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java > javafx.web/src/main/java/com/sun/webkit/MethodHelper.java > 45 public static Object invoke(Method m, Object obj, Object[] params) > > To avoid 3 ModuleHelper classes, the invoke method can take > the callerModule argument to replace this line: > 56 final Module thisModule = MethodHelper.class.getModule(); > I'm fairly certain that won't work. Module::addOpens is caller sensitive and will only work when called from the module in question. > javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java > There are a few other public methods which I think JavaFX doesn?t > need and can be removed. > Yes, I could do this to reduce the public footprint of the class. To minimize the diffs between the original and our copy, I might just comment out the "public". That would also make it easier to add them back in a future version (e.g., to eventually get rid of all dependency on sun.reflect.misc). Thoughts? -- Kevin > Mandy From mandy.chung at oracle.com Tue May 2 21:27:54 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 14:27:54 -0700 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: <08f27f02-2585-9544-7b70-fb55d9062dc2@gmail.com> References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> <08f27f02-2585-9544-7b70-fb55d9062dc2@gmail.com> Message-ID: <9D028093-0C38-438A-9634-837718F87890@oracle.com> Hi Peter, Looking at it again, you are right that no need to skip inflation. The change is simplified. I have verified with and without -Dsun.reflect.noInflation=true. http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8020801/webrev.01/ thanks Mandy > On May 2, 2017, at 1:17 PM, Peter Levart wrote: > > > On 05/02/2017 06:56 PM, Mandy Chung wrote: >>> On May 2, 2017, at 3:14 AM, Peter Levart wrote: >>> >>> I don't quite understand the need for bypassing the inflation of native into generated method accessor >> The VM native reflection implementation does not know about this alternate `reflected$XXX` mechanism. No VM change in this patch and so we ensure this be called via the generated bytecode for Method::invoke rather than going through the VM native reflection which reduces startup overhead. > > I don't think VM native reflection implementation needs to know anything about this alternate `reflected$XXX` mechanism. The NativeMethodAccessorImpl is constructed with the Method argument. In case of `reflected$XXX` mechanism, it is given the alternate Method object that points to the alternate method, so NativeMethodAccessorImpl.invoke0 is called with this alternate Method object. It is like reflecting over the alternate method itself, isn't it? > > Am I missing something? > >>> Is DelegatingMethodAccessorImpl/NativeMethodAccessorImpl combo not treated correctly (i.e. skipped) by the Reflection.getCallerClass(), while generated MethodAccessorImpl subclass is? >> As this case is forced not to go through VM reflection support, unless I miss something, DelegatingMethodAccessorImpl/NativeMethodAccessorImpl has no need to be changed. I will double check with the VM runtime team. > > I was asking because I suspected that this might be the reason for skipping DelegatingMethodAccessorImpl/NativeMethodAccessorImpl. But if it is not the reason (and anyway it would be a bug because other @CallerSensitive methods would behave erratically if this was the case), then I still don't see a reason for skipping DelegatingMethodAccessorImpl/NativeMethodAccessorImpl and proceeding directly with generated method accessor. > > Regards, Peter > >> Mandy >> > From mandy.chung at oracle.com Tue May 2 21:45:46 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 May 2017 14:45:46 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5908F8A9.1000205@oracle.com> References: <5907D11D.20609@oracle.com> <5908F8A9.1000205@oracle.com> Message-ID: > On May 2, 2017, at 2:22 PM, Kevin Rushforth wrote: > > Here is the message: > > IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base > It would be better to emit a more informative message e.g. ?javafx.base cannot access class com.foo (in module foo.app). Either exports com.foo unqualifiedly or open com.foo to javafx.base?. Also in MethodUtil::invoke 61 if (!module.isExported(packageName)) { You can do this check only if module.isNamed. > It is roughly the same message that any similar illegal access would generate (e.g., we get similar error messages when FXML tries to call setAccessible for a field annotated with @FXML if the module is not "open" to javafx.fxml). > >> javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java >> javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java >> javafx.web/src/main/java/com/sun/webkit/MethodHelper.java >> 45 public static Object invoke(Method m, Object obj, Object[] params) >> >> To avoid 3 ModuleHelper classes, the invoke method can take >> the callerModule argument to replace this line: >> 56 final Module thisModule = MethodHelper.class.getModule(); >> > > I'm fairly certain that won't work. Module::addOpens is caller sensitive and will only work when called from the module in question. > You are right. Wont?t work. >> javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java >> There are a few other public methods which I think JavaFX doesn?t >> need and can be removed. >> > > Yes, I could do this to reduce the public footprint of the class. To minimize the diffs between the original and our copy, I might just comment out the "public". That would also make it easier to add them back in a future version (e.g., to eventually get rid of all dependency on sun.reflect.misc). Thoughts? I will leave it up to you. Mandy From alex.buckley at oracle.com Tue May 2 22:27:56 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 02 May 2017 15:27:56 -0700 Subject: Java Platform Module System In-Reply-To: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> References: <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> Message-ID: <590907EC.8050009@oracle.com> On 5/2/2017 5:13 AM, Stephan Herrmann wrote: > Thanks, Alex, for promising improvements in various places of the spec. Re: Multiple packages with the same name can be "visible" (helpful terminology for program analysis) but exactly one of these packages must be identified as the meaning of the name. First, let's define what we want: 7.4.3 Package Observability and Visibility ... A package is _uniquely visible_ to a module M if and only if either (i) an ordinary compilation unit associated with M contains a declaration of the package, and M does not read any other module that exports the package to M; or (ii) no ordinary compilation unit associated with M contains a declaration of the package, and M reads exactly one other module that exports the package to M. Then we can pick the "right" package to go with the name: 6.5.3.1 Simple Package Names If a package name consists of a single Identifier, then the identifier must occur in the scope of exactly one declaration of a top level package with this name (?6.3), and that package must be uniquely visible to the current module (?7.4.3), or a compile-time error occurs. The meaning of the package name is that package. 6.5.3.2 Qualified Package Names If a package name is of the form Q.Id, then Q must also be a package name. The package name Q.Id names a package that is the member named Id within the package named by Q. If Q.Id does not name a package that is uniquely visible to the current module (?7.4.3), then a compile-time error occurs. (Note that 6.5.3.2 did not, and does not, recurse into 6.5.3.1 when the Q in the qualified name Q.Id is simple. 6.5.3.1 relies on scope, which means it applies only when someone has a declaration of a top level package, but plainly 6.5.3.2 has always been able to give meaning to the name P.Q based on a declaration of 'package P.Q;' -- no-one needed to declare a top level 'package P;'. The 6.5.3.2 phrase "Q must also be a package name" in JLS8 was a trailer for the phrase about Q being an observable package, and in JLS9 is a trailer for the phrase about Q.Id being a uniquely visible package.) > I recall several threads on this list ending in you saying "still being > clarified" [1][2]. Are those issues settled by now and just need to be > penned down? Otherwise it would be very helpful just to see the list of > open questions, so we don't bang our heads against walls that are still > subject to change (I'm not speaking about the general "Issue Summary", > but s.t. more focused on JLS and its dependencies in JPMS spec). > > Looking forward to an updated spec version, allowing us to double check > if those changes raise any follow-up questions, > Stephan > > [1] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-January/010866.html This was about multiple packages with the name P being visible at once. So that's covered by the "uniquely visible" invocation in package names. > [2] > http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011544.html The open question there was about multiple modules, directly required by a root module, containing the same package, but without any exports that would cause any module to see a split package. This module graph is legal, so it will resolve in JPMS; the JLS defers to that. javac gives a lint warning (on-by-default, suppressible) if the modules being compiled cannot be mapped to the boot layer. Alex From alex.buckley at oracle.com Tue May 2 22:39:02 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 02 May 2017 15:39:02 -0700 Subject: Java Platform Module System In-Reply-To: References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de>, <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> Message-ID: <59090A86.9070604@oracle.com> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: > Chapter 2 in [1] describes context-free grammars. The addition to "3.9 > Keywords" defines "restricted keywords", which prevent the grammar for > ModuleDeclaration from being context-free. This prevents compilers from > using common parser generators, since those typically only support > context-free grammars. The lexical/syntactic grammar split defined in > chapter 2 is not of much use for actual implementations of > module-info.java parsers. > The spec at least needs to point out that the given grammar for > ModuleDeclaration is not actually context-free. The syntactic grammar in JLS8 was not context-free either; the opening line of Chapter 2 has been false for years. For JLS9, I will remove the claim that the lexical and syntactic grammars are context-free, and perhaps a future JLS can discuss the difficulties in parsing the syntactic grammar. Alex From mandy.chung at oracle.com Wed May 3 00:21:23 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Wed, 03 May 2017 00:21:23 +0000 Subject: hg: jigsaw/jake/jdk: Add @apiNote in ClassLoader::getPackage(s) about platform class loader Message-ID: <201705030021.v430LOPT015428@aojmv0008.oracle.com> Changeset: fa1e20c4e6c1 Author: mchung Date: 2017-05-02 17:20 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/fa1e20c4e6c1 Add @apiNote in ClassLoader::getPackage(s) about platform class loader ! src/java.base/share/classes/java/lang/ClassLoader.java From peter.levart at gmail.com Wed May 3 08:33:51 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 3 May 2017 10:33:51 +0200 Subject: Review Request: JDK-8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke In-Reply-To: <9D028093-0C38-438A-9634-837718F87890@oracle.com> References: <29B7C781-3834-4603-95B3-8900644EDA71@oracle.com> <59f18652-cb90-bb90-e316-c0efb70fb173@gmail.com> <08f27f02-2585-9544-7b70-fb55d9062dc2@gmail.com> <9D028093-0C38-438A-9634-837718F87890@oracle.com> Message-ID: <2985685f-eabf-41ed-804f-2a2a7a71f9c5@gmail.com> Hi Mandy, On 05/02/2017 11:27 PM, Mandy Chung wrote: > Hi Peter, > > Looking at it again, you are right that no need to skip inflation. The > change > is simplified. I have verified with and without > -Dsun.reflect.noInflation=true. > > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8020801/webrev.01/ > Looks good. The following line is not needed now in ReflectionFactory (but is harmless): 182 boolean noInflation = ReflectionFactory.noInflation; Regards, Peter > > thanks > Mandy > >> On May 2, 2017, at 1:17 PM, Peter Levart > > wrote: >> >> >> On 05/02/2017 06:56 PM, Mandy Chung wrote: >>>> On May 2, 2017, at 3:14 AM, Peter Levart wrote: >>>> >>>> I don't quite understand the need for bypassing the inflation of native into generated method accessor >>> The VM native reflection implementation does not know about this alternate `reflected$XXX` mechanism. No VM change in this patch and so we ensure this be called via the generated bytecode for Method::invoke rather than going through the VM native reflection which reduces startup overhead. >> >> I don't think VM native reflection implementation needs to know >> anything about this alternate `reflected$XXX` mechanism. The >> NativeMethodAccessorImpl is constructed with the Method argument. In >> case of `reflected$XXX` mechanism, it is given the alternate Method >> object that points to the alternate method, so >> NativeMethodAccessorImpl.invoke0 is called with this alternate Method >> object. It is like reflecting over the alternate method itself, isn't it? >> >> Am I missing something? >> >>>> Is DelegatingMethodAccessorImpl/NativeMethodAccessorImpl combo not treated correctly (i.e. skipped) by the Reflection.getCallerClass(), while generated MethodAccessorImpl subclass is? >>> As this case is forced not to go through VM reflection support, unless I miss something, DelegatingMethodAccessorImpl/NativeMethodAccessorImpl has no need to be changed. I will double check with the VM runtime team. >> >> I was asking because I suspected that this might be the reason for >> skipping DelegatingMethodAccessorImpl/NativeMethodAccessorImpl. But >> if it is not the reason (and anyway it would be a bug because other >> @CallerSensitive methods would behave erratically if this was the >> case), then I still don't see a reason for skipping >> DelegatingMethodAccessorImpl/NativeMethodAccessorImpl and proceeding >> directly with generated method accessor. >> >> Regards, Peter >> >>> Mandy >>> >> > From harold.seigel at oracle.com Wed May 3 12:25:22 2017 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 May 2017 08:25:22 -0400 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <807ea92f-9bd5-8b62-781b-efb5f68dd33a@oracle.com> Hi Alan, The hotspot changes look good. Thanks, Harold On 5/1/2017 4:28 PM, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From kevin.rushforth at oracle.com Wed May 3 14:48:43 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 03 May 2017 07:48:43 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> <5908F8A9.1000205@oracle.com> Message-ID: <5909EDCB.3030808@oracle.com> OK, I'll create a more informative message. I think it will be more clear in the message to just say that it needs to "open" the package to javafx.base, since that would be the recommendation for a package that isn't already exported unconditionally. I'll send out a new webrev this morning with all feedback incorporated. -- Kevin Mandy Chung wrote: >> On May 2, 2017, at 2:22 PM, Kevin Rushforth wrote: >> >> Here is the message: >> >> IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base >> >> > > It would be better to emit a more informative message e.g. ?javafx.base cannot access class com.foo (in module foo.app). Either exports com.foo unqualifiedly or open com.foo to javafx.base?. Also in MethodUtil::invoke > 61 if (!module.isExported(packageName)) { > You can do this check only if module.isNamed. > > >> It is roughly the same message that any similar illegal access would generate (e.g., we get similar error messages when FXML tries to call setAccessible for a field annotated with @FXML if the module is not "open" to javafx.fxml). >> >> >>> javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java >>> javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java >>> javafx.web/src/main/java/com/sun/webkit/MethodHelper.java >>> 45 public static Object invoke(Method m, Object obj, Object[] params) >>> >>> To avoid 3 ModuleHelper classes, the invoke method can take >>> the callerModule argument to replace this line: >>> 56 final Module thisModule = MethodHelper.class.getModule(); >>> >>> >> I'm fairly certain that won't work. Module::addOpens is caller sensitive and will only work when called from the module in question. >> >> > > You are right. Wont?t work. > >>> javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java >>> There are a few other public methods which I think JavaFX doesn?t >>> need and can be removed. >>> >>> >> Yes, I could do this to reduce the public footprint of the class. To minimize the diffs between the original and our copy, I might just comment out the "public". That would also make it easier to add them back in a future version (e.g., to eventually get rid of all dependency on sun.reflect.misc). Thoughts? >> > > I will leave it up to you. > Mandy > > From kevin.rushforth at oracle.com Wed May 3 14:57:08 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 03 May 2017 07:57:08 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5909EDCB.3030808@oracle.com> References: <5907D11D.20609@oracle.com> <5908F8A9.1000205@oracle.com> <5909EDCB.3030808@oracle.com> Message-ID: <5909EFC4.8070006@oracle.com> Forgot to respond to this: > Also in MethodUtil::invoke > 61 if (!module.isExported(packageName)) { > You can do this check only if module.isNamed. No, this check will work fine on an unnamed module. Module::isExported and Module::isOpen always return true for Module::unnamed. -- Kevin Kevin Rushforth wrote: > OK, I'll create a more informative message. I think it will be more > clear in the message to just say that it needs to "open" the package > to javafx.base, since that would be the recommendation for a package > that isn't already exported unconditionally. I'll send out a new > webrev this morning with all feedback incorporated. > > -- Kevin > > > Mandy Chung wrote: >>> On May 2, 2017, at 2:22 PM, Kevin Rushforth >>> wrote: >>> >>> Here is the message: >>> >>> IllegalAccessException: class com.sun.javafx.property.MethodHelper >>> cannot access class com.foo (in module foo.app) because module >>> foo.app does not open com.foo to javafx.base >>> >> >> It would be better to emit a more informative message e.g. >> ?javafx.base cannot access class com.foo (in module foo.app). Either >> exports com.foo unqualifiedly or open com.foo to javafx.base?. Also >> in MethodUtil::invoke >> 61 if (!module.isExported(packageName)) { >> You can do this check only if module.isNamed. >> >> >>> It is roughly the same message that any similar illegal access would >>> generate (e.g., we get similar error messages when FXML tries to >>> call setAccessible for a field annotated with @FXML if the module is >>> not "open" to javafx.fxml). >>> >>> >>>> javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java >>>> javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java >>>> javafx.web/src/main/java/com/sun/webkit/MethodHelper.java >>>> 45 public static Object invoke(Method m, Object obj, Object[] >>>> params) >>>> >>>> To avoid 3 ModuleHelper classes, the invoke method can take >>>> the callerModule argument to replace this line: 56 final >>>> Module thisModule = MethodHelper.class.getModule(); >>>> >>> I'm fairly certain that won't work. Module::addOpens is caller >>> sensitive and will only work when called from the module in question. >>> >>> >> >> You are right. Wont?t work. >> >>>> javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java >>>> There are a few other public methods which I think JavaFX doesn?t >>>> need and can be removed. >>>> >>> Yes, I could do this to reduce the public footprint of the class. To >>> minimize the diffs between the original and our copy, I might just >>> comment out the "public". That would also make it easier to add them >>> back in a future version (e.g., to eventually get rid of all >>> dependency on sun.reflect.misc). Thoughts? >>> >> >> I will leave it up to you. >> Mandy >> >> From mandy.chung at oracle.com Wed May 3 16:00:51 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Wed, 03 May 2017 16:00:51 +0000 Subject: hg: jigsaw/jake/jdk: Minor tweak to the api note Message-ID: <201705031600.v43G0pgV028213@aojmv0008.oracle.com> Changeset: 654588131ea2 Author: mchung Date: 2017-05-03 08:59 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/654588131ea2 Minor tweak to the api note ! src/java.base/share/classes/java/lang/ClassLoader.java From alex.buckley at oracle.com Wed May 3 17:46:54 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 03 May 2017 10:46:54 -0700 Subject: Java Platform Module System In-Reply-To: <59090A86.9070604@oracle.com> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de>, <20170424091831.557135740@eggemoggin.niobe.net> <58FEAB69.2080008@oracle.com> <58FF812E.1010006@oracle.com> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <59090A86.9070604@oracle.com> Message-ID: <590A178E.6010908@oracle.com> On 5/2/2017 3:39 PM, Alex Buckley wrote: > On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: >> Chapter 2 in [1] describes context-free grammars. The addition to "3.9 >> Keywords" defines "restricted keywords", which prevent the grammar for >> ModuleDeclaration from being context-free. This prevents compilers from >> using common parser generators, since those typically only support >> context-free grammars. The lexical/syntactic grammar split defined in >> chapter 2 is not of much use for actual implementations of >> module-info.java parsers. >> The spec at least needs to point out that the given grammar for >> ModuleDeclaration is not actually context-free. > > The syntactic grammar in JLS8 was not context-free either; the opening > line of Chapter 2 has been false for years. For JLS9, I will remove the > claim that the lexical and syntactic grammars are context-free, and > perhaps a future JLS can discuss the difficulties in parsing the Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is what I should have said the grammar hasn't been for a long time. Alex From forax at univ-mlv.fr Wed May 3 18:55:29 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 3 May 2017 20:55:29 +0200 (CEST) Subject: Java Platform Module System In-Reply-To: <590A178E.6010908@oracle.com> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <59090A86.9070604@oracle.com> <590A178E.6010908@oracle.com> Message-ID: <725214201.1675310.1493837729692.JavaMail.zimbra@u-pem.fr> It's context-free because a context free grammar defined its input in term of terminals and the theory do not say how to map a token to a terminal. Jay is right that it requires to use either some specific parser generator like Tatoo [1] the one i've written 10 years ago (because i wanted the tool to help me to extend a grammar easily) or to modify an existing parser generator so the parser can send the production state to the lexer which will enable/disable the automata that recognize the associated keywords . R?mi [1] http://dl.acm.org/citation.cfm?id=1168057 ----- Mail original ----- > De: "Alex Buckley" > ?: "Jayaprakash Arthanareeswaran" , "Dan Smith" , "Brian Goetz" > > Cc: jigsaw-dev at openjdk.java.net > Envoy?: Mercredi 3 Mai 2017 19:46:54 > Objet: Re: Java Platform Module System > On 5/2/2017 3:39 PM, Alex Buckley wrote: >> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: >>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9 >>> Keywords" defines "restricted keywords", which prevent the grammar for >>> ModuleDeclaration from being context-free. This prevents compilers from >>> using common parser generators, since those typically only support >>> context-free grammars. The lexical/syntactic grammar split defined in >>> chapter 2 is not of much use for actual implementations of >>> module-info.java parsers. >>> The spec at least needs to point out that the given grammar for >>> ModuleDeclaration is not actually context-free. >> >> The syntactic grammar in JLS8 was not context-free either; the opening >> line of Chapter 2 has been false for years. For JLS9, I will remove the >> claim that the lexical and syntactic grammars are context-free, and >> perhaps a future JLS can discuss the difficulties in parsing the > > Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and > JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is > what I should have said the grammar hasn't been for a long time. > > Alex From stephan.herrmann at berlin.de Wed May 3 21:31:14 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Wed, 3 May 2017 23:31:14 +0200 Subject: Java Platform Module System In-Reply-To: <725214201.1675310.1493837729692.JavaMail.zimbra@u-pem.fr> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> <1b984d15-2056-5860-a0b0-40d85edffe42@berlin.de> <5903CF7D.9060803@oracle.com> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <59090A86.9070604@oracle.com> <590A178E.6010908@oracle.com> <725214201.1675310.1493837729692.JavaMail.zimbra@u-pem.fr> Message-ID: <6845fd14-35b8-a1fc-5231-2d09cfdd12d2@berlin.de> On 03.05.2017 20:55, Remi Forax wrote: > It's context-free because a context free grammar defined its input in term of > terminals and the theory do not say how to map a token to a terminal. > > Jay is right that it requires to use either some specific parser generator > like Tatoo [1] the one i've written 10 years ago (because i wanted the tool to > help me to extend a grammar easily) or to modify an existing parser generator so > the parser can send the production state to the lexer which will enable/disable > the automata that recognize the associated keywords . Just feeding parser state into the Lexer doesn't cut it for Java 9, because the classification keyword / identifier cannot be made at the time when the stream passes the Lexer. Let me remind you of this example: module foo { exports transitive How should the poor lexer recognize in this situation that transitive is an identifier (sic) (if you complete the text accordingly)? Aside from specific heuristics (which are not available to any parser generator), we only know about this classification after the parser has matched an entire declaration. I'm not even sure that theory has a name for this kind of grammar. Maybe we should speak of a constraint solver rather than a parser. Stephan > > R?mi > > [1] http://dl.acm.org/citation.cfm?id=1168057 > > ----- Mail original ----- >> De: "Alex Buckley" >> ?: "Jayaprakash Arthanareeswaran" , "Dan Smith" , "Brian Goetz" >> >> Cc: jigsaw-dev at openjdk.java.net >> Envoy?: Mercredi 3 Mai 2017 19:46:54 >> Objet: Re: Java Platform Module System > >> On 5/2/2017 3:39 PM, Alex Buckley wrote: >>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: >>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9 >>>> Keywords" defines "restricted keywords", which prevent the grammar for >>>> ModuleDeclaration from being context-free. This prevents compilers from >>>> using common parser generators, since those typically only support >>>> context-free grammars. The lexical/syntactic grammar split defined in >>>> chapter 2 is not of much use for actual implementations of >>>> module-info.java parsers. >>>> The spec at least needs to point out that the given grammar for >>>> ModuleDeclaration is not actually context-free. >>> >>> The syntactic grammar in JLS8 was not context-free either; the opening >>> line of Chapter 2 has been false for years. For JLS9, I will remove the >>> claim that the lexical and syntactic grammars are context-free, and >>> perhaps a future JLS can discuss the difficulties in parsing the >> >> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and >> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is >> what I should have said the grammar hasn't been for a long time. >> >> Alex From forax at univ-mlv.fr Wed May 3 22:06:56 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 4 May 2017 00:06:56 +0200 (CEST) Subject: Java Platform Module System In-Reply-To: <6845fd14-35b8-a1fc-5231-2d09cfdd12d2@berlin.de> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <59090A86.9070604@oracle.com> <590A178E.6010908@oracle.com> <725214201.1675310.1493837729692.JavaMail.zimbra@u-pem.fr> <6845fd14-35b8-a1fc-5231-2d09cfdd12d2@berlin.de> Message-ID: <238566754.1726064.1493849216527.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stephan Herrmann" > ?: jigsaw-dev at openjdk.java.net, "Remi Forax" , "Alex Buckley" > Cc: "Brian Goetz" , "Dan Smith" > Envoy?: Mercredi 3 Mai 2017 23:31:14 > Objet: Re: Java Platform Module System > On 03.05.2017 20:55, Remi Forax wrote: > > It's context-free because a context free grammar defined its input in term of > > terminals and the theory do not say how to map a token to a terminal. > > > > Jay is right that it requires to use either some specific parser generator > > like Tatoo [1] the one i've written 10 years ago (because i wanted the tool to > > help me to extend a grammar easily) or to modify an existing parser generator so > > the parser can send the production state to the lexer which will enable/disable > > the automata that recognize the associated keywords . > > Just feeding parser state into the Lexer doesn't cut it for Java 9, > because the classification keyword / identifier cannot be made at > the time when the stream passes the Lexer. No, it's done between the lexer and the parser > Let me remind you of this example: > module foo { exports transitive > How should the poor lexer recognize in this situation that transitive > is an identifier (sic) (if you complete the text accordingly)? There is a simple solution, consider module, requires, etc as keyword in the lexer, and when the keyword is sent to the parser, downgrade it to an identifier if you are not at the right dotted production. It's easy to implement if your lexer/parser is non blocking, i.e if you push a bytebuffer to the lexer and the lexer push terminals to the parser. the other solution used by Tatoo is to instead of having one giant automata that recognize all tokens, works with a list automata that recognized each token and activate them or not depending on the parser state. > Aside from specific heuristics > (which are not available to any parser generator), any but one :) > we only know about this classification after the parser has matched > an entire declaration. so i suppose your parser is LR, you know the classification from the dotted production just before the terminal is about to be recognized. when you construct the LR table, you know for each dotted production what are the terminals that can appear so the parser generator can keep these info in a side table and during the parsing, from the parser state, find which terminals can be recognized. > I'm not even sure that theory has a name for this kind of grammar. > Maybe we should speak of a constraint solver rather than a parser. no need to have a constraint solver here, you need to export the terminals that will lead to a shift or a reduce for any LR states. > > Stephan > R?mi >> >> R?mi >> >> [1] http://dl.acm.org/citation.cfm?id=1168057 >> >> ----- Mail original ----- >>> De: "Alex Buckley" >>> ?: "Jayaprakash Arthanareeswaran" , "Dan Smith" >>> , "Brian Goetz" >>> >>> Cc: jigsaw-dev at openjdk.java.net >>> Envoy?: Mercredi 3 Mai 2017 19:46:54 >>> Objet: Re: Java Platform Module System >> >>> On 5/2/2017 3:39 PM, Alex Buckley wrote: >>>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: >>>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9 >>>>> Keywords" defines "restricted keywords", which prevent the grammar for >>>>> ModuleDeclaration from being context-free. This prevents compilers from >>>>> using common parser generators, since those typically only support >>>>> context-free grammars. The lexical/syntactic grammar split defined in >>>>> chapter 2 is not of much use for actual implementations of >>>>> module-info.java parsers. >>>>> The spec at least needs to point out that the given grammar for >>>>> ModuleDeclaration is not actually context-free. >>>> >>>> The syntactic grammar in JLS8 was not context-free either; the opening >>>> line of Chapter 2 has been false for years. For JLS9, I will remove the >>>> claim that the lexical and syntactic grammars are context-free, and >>>> perhaps a future JLS can discuss the difficulties in parsing the >>> >>> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and >>> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is >>> what I should have said the grammar hasn't been for a long time. >>> > >> Alex From stephan.herrmann at berlin.de Wed May 3 23:05:51 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Thu, 4 May 2017 01:05:51 +0200 Subject: Java Platform Module System In-Reply-To: <238566754.1726064.1493849216527.JavaMail.zimbra@u-pem.fr> References: <0e5a9784-1a22-3f13-a292-4f9314372188@berlin.de> <5c2e7b9c-c4b1-e618-49bc-505791c3eed4@berlin.de> <59079667.2060308@oracle.com> <59090A86.9070604@oracle.com> <590A178E.6010908@oracle.com> <725214201.1675310.1493837729692.JavaMail.zimbra@u-pem.fr> <6845fd14-35b8-a1fc-5231-2d09cfdd12d2@berlin.de> <238566754.1726064.1493849216527.JavaMail.zimbra@u-pem.fr> Message-ID: R?mi, of course there are hacks & tricks to parse such a language. My main point is: it's outside any established concepts and technology. And: if you happen to maintain a tool where recovery after syntax errors and code completion and tons of other features are based on a given parser generator, any deviation from such established concepts really hurts. We are not in the position to exchange or rewrite the parser generator, we have to code around it, which inevitably is a messy endeavor. We're doing it, don't worry. It would just be fair to spell out what kind of grammar this is. I don't have a name for it. The best description I could come up with so far is: "restricted keywords are keywords when it helps to interpret them as keywords". Stephan On 04.05.2017 00:06, forax at univ-mlv.fr wrote: > ----- Mail original ----- >> De: "Stephan Herrmann" >> ?: jigsaw-dev at openjdk.java.net, "Remi Forax" , "Alex Buckley" >> Cc: "Brian Goetz" , "Dan Smith" >> Envoy?: Mercredi 3 Mai 2017 23:31:14 >> Objet: Re: Java Platform Module System > >> On 03.05.2017 20:55, Remi Forax wrote: >>> It's context-free because a context free grammar defined its input in term of >>> terminals and the theory do not say how to map a token to a terminal. >>> >>> Jay is right that it requires to use either some specific parser generator >>> like Tatoo [1] the one i've written 10 years ago (because i wanted the tool to >>> help me to extend a grammar easily) or to modify an existing parser generator so >>> the parser can send the production state to the lexer which will enable/disable >>> the automata that recognize the associated keywords . >> >> Just feeding parser state into the Lexer doesn't cut it for Java 9, >> because the classification keyword / identifier cannot be made at >> the time when the stream passes the Lexer. > > No, it's done between the lexer and the parser > >> Let me remind you of this example: >> module foo { exports transitive >> How should the poor lexer recognize in this situation that transitive >> is an identifier (sic) (if you complete the text accordingly)? > > There is a simple solution, consider module, requires, etc as keyword in the lexer, and when the keyword is sent to the parser, downgrade it to an identifier if you are not at the right dotted production. > > It's easy to implement if your lexer/parser is non blocking, i.e if you push a bytebuffer to the lexer and the lexer push terminals to the parser. > > the other solution used by Tatoo is to instead of having one giant automata that recognize all tokens, works with a list automata that recognized each token and activate them or not depending on the parser state. > >> Aside from specific heuristics >> (which are not available to any parser generator), > > any but one :) > >> we only know about this classification after the parser has matched >> an entire declaration. > > so i suppose your parser is LR, you know the classification from the dotted production just before the terminal is about to be recognized. > when you construct the LR table, you know for each dotted production what are the terminals that can appear so the parser generator can keep these info in a side table and during the parsing, from the parser state, find which terminals can be recognized. > >> I'm not even sure that theory has a name for this kind of grammar. >> Maybe we should speak of a constraint solver rather than a parser. > > no need to have a constraint solver here, you need to export the terminals that will lead to a shift or a reduce for any LR states. > >> >> Stephan >> > > R?mi > >>> >>> R?mi >>> >>> [1] http://dl.acm.org/citation.cfm?id=1168057 >>> >>> ----- Mail original ----- >>>> De: "Alex Buckley" >>>> ?: "Jayaprakash Arthanareeswaran" , "Dan Smith" >>>> , "Brian Goetz" >>>> >>>> Cc: jigsaw-dev at openjdk.java.net >>>> Envoy?: Mercredi 3 Mai 2017 19:46:54 >>>> Objet: Re: Java Platform Module System >>> >>>> On 5/2/2017 3:39 PM, Alex Buckley wrote: >>>>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote: >>>>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9 >>>>>> Keywords" defines "restricted keywords", which prevent the grammar for >>>>>> ModuleDeclaration from being context-free. This prevents compilers from >>>>>> using common parser generators, since those typically only support >>>>>> context-free grammars. The lexical/syntactic grammar split defined in >>>>>> chapter 2 is not of much use for actual implementations of >>>>>> module-info.java parsers. >>>>>> The spec at least needs to point out that the given grammar for >>>>>> ModuleDeclaration is not actually context-free. >>>>> >>>>> The syntactic grammar in JLS8 was not context-free either; the opening >>>>> line of Chapter 2 has been false for years. For JLS9, I will remove the >>>>> claim that the lexical and syntactic grammars are context-free, and >>>>> perhaps a future JLS can discuss the difficulties in parsing the >>>> >>>> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and >>>> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is >>>> what I should have said the grammar hasn't been for a long time. >>>> >>>> Alex From kevin.rushforth at oracle.com Wed May 3 23:30:07 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 03 May 2017 16:30:07 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5907D11D.20609@oracle.com> References: <5907D11D.20609@oracle.com> Message-ID: <590A67FF.40408@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8177566 Here is the updated webrev with (I hope) all comments addressed: http://cr.openjdk.java.net/~kcr/8177566/webrev.01/complete-webrev/ For those who reviewed the earlier webrev, I have prepared delta webrevs. * Delta webrev for the fix itself (just a slight change to the error message, plus I hid the unused public methods in MethodUtil) : http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-fix-only.00/ * No changes in the tests * Delta webrev for the doc changes: http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-doc-only.00/ * The sparse javadocs are also updated here: http://cr.openjdk.java.net/~kcr/8177566/webrev.01/javadoc/ -- Kevin Kevin Rushforth wrote: > This review is being cross-posted to both openjfx-dev and jigsaw-dev. > > Please review the proposed fix for: > > https://bugs.openjdk.java.net/browse/JDK-8177566 > http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ > > Details of the fix as well as notes to reviewers are in the bug report > [1] (e.g., I've also generated separate webrevs for the fix itself, > the doc changes, and the test changes). > > -- Kevin > > [1] > https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 > > From mandy.chung at oracle.com Thu May 4 00:05:28 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 May 2017 17:05:28 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <590A67FF.40408@oracle.com> References: <5907D11D.20609@oracle.com> <590A67FF.40408@oracle.com> Message-ID: Looks good. "Deploying an Application as a Module? section is duplicated in several JavaBean*Property classes. One alternative is to move it to the package summary. I have no objection to leave it as is. Mandy > On May 3, 2017, at 4:30 PM, Kevin Rushforth wrote: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8177566 > > Here is the updated webrev with (I hope) all comments addressed: > > http://cr.openjdk.java.net/~kcr/8177566/webrev.01/complete-webrev/ > > For those who reviewed the earlier webrev, I have prepared delta webrevs. > > * Delta webrev for the fix itself (just a slight change to the error message, plus I hid the unused public methods in MethodUtil) : > > http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-fix-only.00/ > > * No changes in the tests > > * Delta webrev for the doc changes: > > http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-doc-only.00/ > > * The sparse javadocs are also updated here: > > http://cr.openjdk.java.net/~kcr/8177566/webrev.01/javadoc/ > > -- Kevin > > > Kevin Rushforth wrote: >> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >> >> Please review the proposed fix for: >> >> https://bugs.openjdk.java.net/browse/JDK-8177566 >> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >> >> Details of the fix as well as notes to reviewers are in the bug report [1] (e.g., I've also generated separate webrevs for the fix itself, the doc changes, and the test changes). >> >> -- Kevin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 >> From kevin.rushforth at oracle.com Thu May 4 00:14:14 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 03 May 2017 17:14:14 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> <590A67FF.40408@oracle.com> Message-ID: <590A7256.30406@oracle.com> Mandy Chung wrote: > Looks good. > Thank you for your help on this and for your review. > "Deploying an Application as a Module? section is duplicated in several > JavaBean*Property classes. One alternative is to move it to the package > summary. I have no objection to leave it as is. > I think I'll keep it as is for the JavaBean*Property classes. Especially since we don't have anything currently in the package summary. There might be other detailed information that could go there, too. -- Kevin > Mandy > > > >> On May 3, 2017, at 4:30 PM, Kevin Rushforth wrote: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177566 >> >> Here is the updated webrev with (I hope) all comments addressed: >> >> http://cr.openjdk.java.net/~kcr/8177566/webrev.01/complete-webrev/ >> >> For those who reviewed the earlier webrev, I have prepared delta webrevs. >> >> * Delta webrev for the fix itself (just a slight change to the error message, plus I hid the unused public methods in MethodUtil) : >> >> http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-fix-only.00/ >> >> * No changes in the tests >> >> * Delta webrev for the doc changes: >> >> http://cr.openjdk.java.net/~kcr/8177566/webrev.01/delta-doc-only.00/ >> >> * The sparse javadocs are also updated here: >> >> http://cr.openjdk.java.net/~kcr/8177566/webrev.01/javadoc/ >> >> -- Kevin >> >> >> Kevin Rushforth wrote: >> >>> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >>> >>> Please review the proposed fix for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8177566 >>> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >>> >>> Details of the fix as well as notes to reviewers are in the bug report [1] (e.g., I've also generated separate webrevs for the fix itself, the doc changes, and the test changes). >>> >>> -- Kevin >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 >>> >>> > > From serguei.spitsyn at oracle.com Thu May 4 00:29:06 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 3 May 2017 17:29:06 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: Hi Alan, I reviewed the Hotspot changes and the Jdk changes related to java.instrument and jdk.attch (including a fragment in the LauncherHelper.java). The fixes look good to me. Thanks, Serguei On 5/1/17 13:28, Alan Bateman wrote: > As I mentioned in another thread, we need to get the changes > accumulated in jake to jdk9/dev. > > JDK-8178380 [1] has the summary of the changes that have accumulated > since the last refresh. > > One note on the hotspot repo is that the changes to dynamically > augment the platform modules run into JDK-8178604 when testing with an > exploded build. So there is a partial fix for this in jake. Harold has > the more complete fix in review on hotspot-runtime-dev for JDK 10. > > The webrevs with the changes is here: > http://cr.openjdk.java.net/~alanb/8178380/1/ > > -Alan > > [1] https://bugs.openjdk.java.net/browse/JDK-8178380 > From mandy.chung at oracle.com Thu May 4 02:04:06 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 04 May 2017 02:04:06 +0000 Subject: hg: jigsaw/jake/jdk: 2 new changesets Message-ID: <201705040204.v44247dM001177@aojmv0008.oracle.com> Changeset: f1ee71109b07 Author: mchung Date: 2017-05-03 19:01 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/f1ee71109b07 Fix IntegrationTest.java test to check JAVA_FULL_VERSION ! test/tools/jlink/IntegrationTest.java Changeset: 00902040a130 Author: mchung Date: 2017-05-03 19:02 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/00902040a130 --show-module-resolution is not detected properly ! src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java + test/tools/launcher/modules/showmoduleresolution/ShowModuleResolutionTest.java From mandy.chung at oracle.com Thu May 4 02:08:21 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 May 2017 19:08:21 -0700 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: > On May 1, 2017, at 1:28 PM, Alan Bateman wrote: > > http://cr.openjdk.java.net/~alanb/8178380/2/ I reviewed all repos in the new version. src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java 148 if (System.getProperty("jdk.module.minimumBoot") != null) { This property can be removed after read the value, if present. 287 if (propValue != null && Boolean.getBoolean(propValue)) It should use Boolean.parseBoolean. I have fixed the above injake and also added a new test to verify -?show-module-resolution option. Otherwise, all looks good. Mandy From Alan.Bateman at oracle.com Thu May 4 06:23:28 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 May 2017 07:23:28 +0100 Subject: 8178380: Module system implementation refresh (5/2017 update) In-Reply-To: References: Message-ID: <6c37b392-01c5-c881-4fe5-69fd5ef7c9d8@oracle.com> On 04/05/2017 03:08, Mandy Chung wrote: > > > src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java > > 148 if (System.getProperty("jdk.module.minimumBoot") != null) { > This property can be removed after read the value, if present. It doesn't need to because VM terminates immediately. > > I have fixed the above injake and also added a new test to verify > -?show-module-resolution option. > Okay, I'll check these out - thanks. From alan.bateman at oracle.com Thu May 4 10:54:20 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 10:54:20 +0000 Subject: hg: jigsaw/jake/nashorn: 3 new changesets Message-ID: <201705041054.v44AsKmj028490@aojmv0008.oracle.com> Changeset: a3022cc65b17 Author: jjg Date: 2017-04-26 10:56 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/a3022cc65b17 8179304: Fix HTML 5 errors in jdk.scripting.nashorn and jdk.dynalink module Reviewed-by: sundar, jlaskey, hannesw ! src/jdk.dynalink/share/classes/jdk/dynalink/StandardOperation.java ! src/jdk.dynalink/share/classes/jdk/dynalink/linker/GuardingTypeConverterFactory.java ! src/jdk.scripting.nashorn/share/classes/jdk/nashorn/api/tree/TreeVisitor.java Changeset: 0f81cde5a1f7 Author: lana Date: 2017-04-27 21:16 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/0f81cde5a1f7 Merge Changeset: 9d00d3d26242 Author: alanb Date: 2017-05-04 11:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/9d00d3d26242 Merge From alan.bateman at oracle.com Thu May 4 10:54:20 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 10:54:20 +0000 Subject: hg: jigsaw/jake/langtools: 9 new changesets Message-ID: <201705041054.v44AsKwp028495@aojmv0008.oracle.com> Changeset: 13f457e05af0 Author: smarks Date: 2017-04-25 16:14 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/13f457e05af0 8168444: (jdeprscan) improper handling of primitives and primitive array types Reviewed-by: psandoz, jjg ! src/jdk.jdeps/share/classes/com/sun/tools/jdeprscan/scan/Scan.java ! test/tools/jdeprscan/tests/jdk/jdeprscan/TestLoadExpected.csv + test/tools/jdeprscan/tests/jdk/jdeprscan/TestPrims.csv + test/tools/jdeprscan/tests/jdk/jdeprscan/TestPrims.java Changeset: fff0714129d8 Author: jjg Date: 2017-04-25 18:35 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/fff0714129d8 8179299: Fix HTML 5 errors in java.compiler module 8179300: Fix HTML 5 errors in jdk.compiler module 8179301: Fix HTML 5 errors in jdk.javadoc module 8179303: Fix HTML 5 errors in jdk.jshell module Reviewed-by: darcy ! src/java.compiler/share/classes/javax/lang/model/element/ModuleElement.java ! src/java.compiler/share/classes/javax/tools/JavaFileManager.java ! src/jdk.compiler/share/classes/com/sun/source/doctree/DocTreeVisitor.java ! src/jdk.compiler/share/classes/com/sun/source/tree/LambdaExpressionTree.java ! src/jdk.compiler/share/classes/com/sun/source/tree/TreeVisitor.java ! src/jdk.javadoc/share/classes/com/sun/javadoc/FieldDoc.java ! src/jdk.javadoc/share/classes/com/sun/javadoc/MethodDoc.java ! src/jdk.javadoc/share/classes/com/sun/javadoc/Tag.java ! src/jdk.javadoc/share/classes/com/sun/javadoc/package-info.java ! src/jdk.javadoc/share/classes/jdk/javadoc/doclet/package-info.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/JdiExecutionControlProvider.java Changeset: 6a8ceafed0e8 Author: bpatel Date: 2017-04-26 08:15 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/6a8ceafed0e8 8166306: Broken link for All Packages in java.jnlp module Reviewed-by: jjg, ksrini ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractPackageIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDoclet.java ! test/jdk/javadoc/doclet/testModules/TestModules.java Changeset: 29001eb39fd8 Author: smarks Date: 2017-04-26 15:49 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/29001eb39fd8 8169203: (jdeprscan) eliminate duplicate "can't find class" errors Reviewed-by: jjg ! src/jdk.jdeps/share/classes/com/sun/tools/jdeprscan/scan/Scan.java + test/tools/jdeprscan/tests/jdk/jdeprscan/TestNotFound.java Changeset: af62fa3a3a89 Author: lana Date: 2017-04-27 21:16 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/af62fa3a3a89 Merge Changeset: bc21e5ba6bf1 Author: mcimadamore Date: 2017-04-28 11:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/bc21e5ba6bf1 8178437: remove tools/javac/lambda/speculative/T8177933.java Summary: remove test from test folder and from problem list Reviewed-by: jlahoda ! test/ProblemList.txt - test/tools/javac/lambda/speculative/T8177933.java Changeset: c4129e2ec84f Author: alanb Date: 2017-05-04 07:28 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/c4129e2ec84f 8178380: Module system implementation refresh (5/2017) Reviewed-by: jjg, mchung ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ModuleTarget_attribute.java ! src/jdk.jdeps/share/classes/com/sun/tools/javap/AttributeWriter.java Changeset: b22dee7cea7d Author: alanb Date: 2017-05-04 09:43 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/b22dee7cea7d Merge Changeset: 3abaf27917da Author: alanb Date: 2017-05-04 11:01 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/3abaf27917da Merge ! src/java.compiler/share/classes/javax/lang/model/element/ModuleElement.java ! src/java.compiler/share/classes/javax/tools/JavaFileManager.java ! src/jdk.compiler/share/classes/com/sun/source/tree/TreeVisitor.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractPackageIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDoclet.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ModuleTarget_attribute.java ! src/jdk.jdeps/share/classes/com/sun/tools/javap/AttributeWriter.java ! test/ProblemList.txt ! test/jdk/javadoc/doclet/testModules/TestModules.java - test/tools/javac/lambda/speculative/T8177933.java From alan.bateman at oracle.com Thu May 4 10:54:21 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 10:54:21 +0000 Subject: hg: jigsaw/jake: 11 new changesets Message-ID: <201705041054.v44AsLwb028499@aojmv0008.oracle.com> Changeset: e1cd0e7075ae Author: ksrini Date: 2017-04-25 07:54 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/e1cd0e7075ae 8178725: provide way to link to external documentation Reviewed-by: dholmes, erikj, ihse, jjg ! make/Javadoc.gmk Changeset: 4a25d35d9159 Author: ihse Date: 2017-04-26 14:34 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/4a25d35d9159 8178042: Allow custom taglets Reviewed-by: erikj, mchung ! make/Javadoc.gmk Changeset: 7f290cff26cd Author: lana Date: 2017-04-27 21:16 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/7f290cff26cd Merge Changeset: 55b9ec67de79 Author: ihse Date: 2017-04-28 10:46 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/55b9ec67de79 8179361: specify -javafx option for javadoc command Reviewed-by: erikj ! make/Javadoc.gmk Changeset: a09f8a1f40ef Author: dnsimon Date: 2017-04-27 23:06 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/a09f8a1f40ef 8177845: Need a mechanism to load Graal Reviewed-by: kvn, iveresov, mchung ! make/CompileJavaModules.gmk ! make/common/Modules.gmk Changeset: 143d4c87bc1e Author: erikj Date: 2017-04-28 15:40 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/143d4c87bc1e 8179225: Update graphviz bundle script with up to date build instructions Reviewed-by: tbell, ihse ! make/devkit/createGraphvizBundle.sh Changeset: 481bd01358a9 Author: ihse Date: 2017-05-03 08:46 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/481bd01358a9 8179453: Add a proper SetupProcessMarkdown Reviewed-by: erikj ! common/doc/testing.md ! make/Javadoc.gmk ! make/UpdateBuildDocs.gmk + make/common/ProcessMarkdown.gmk Changeset: 5288abbc445c Author: erikj Date: 2017-05-03 20:49 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/5288abbc445c 8179438: Incremental builds broken on Windows Reviewed-by: tbell, ihse ! make/common/NativeCompilation.gmk Changeset: 0b1d62e12105 Author: alanb Date: 2017-05-04 07:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/0b1d62e12105 8178380: Module system implementation refresh (5/2017) Reviewed-by: alanb Contributed-by: mandy.chung at oracle.com ! common/autoconf/generated-configure.sh ! common/autoconf/platform.m4 ! common/autoconf/spec.gmk.in ! make/CreateJmods.gmk ! make/ReleaseFile.gmk Changeset: 30db903fa442 Author: alanb Date: 2017-05-04 09:43 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/30db903fa442 Merge ! common/autoconf/generated-configure.sh Changeset: cdd7aa3eec8e Author: alanb Date: 2017-05-04 11:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/cdd7aa3eec8e Merge ! common/autoconf/generated-configure.sh ! common/autoconf/platform.m4 ! common/autoconf/spec.gmk.in ! make/CompileJavaModules.gmk ! make/CreateJmods.gmk ! make/Javadoc.gmk ! make/common/Modules.gmk ! make/common/NativeCompilation.gmk From alan.bateman at oracle.com Thu May 4 10:54:25 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 10:54:25 +0000 Subject: hg: jigsaw/jake/hotspot: 6 new changesets Message-ID: <201705041054.v44AsPpj028516@aojmv0008.oracle.com> Changeset: f1cca489e9c6 Author: kbarrett Date: 2017-04-26 22:34 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/f1cca489e9c6 8179084: HotSpot VM fails to start when AggressiveHeap is set Summary: Don't set default ParallelGCThreads when processing AggressiveHeap Reviewed-by: stefank, ehelin ! src/share/vm/runtime/arguments.cpp + test/gc/arguments/TestAggressiveHeap.java Changeset: 537dbd601b34 Author: lana Date: 2017-04-27 21:16 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/537dbd601b34 Merge Changeset: fbb9c8026495 Author: dnsimon Date: 2017-04-28 02:54 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/fbb9c8026495 8177845: Need a mechanism to load Graal Reviewed-by: kvn, iveresov, mchung ! .mx.jvmci/.pydevproject ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/EventProvider.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCICompilerConfig.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotJVMCIRuntime.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotMemoryAccessProviderImpl.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.services/src/jdk/vm/ci/services/JVMCIServiceLocator.java ! src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.services/src/jdk/vm/ci/services/Services.java + src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.services/src/jdk/vm/ci/services/internal/ReflectionAccessJDK.java ! src/jdk.internal.vm.ci/share/classes/module-info.java ! src/jdk.internal.vm.compiler/.mx.graal/.pydevproject ! src/jdk.internal.vm.compiler/.mx.graal/suite.py ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.api.test/src/org/graalvm/compiler/api/test/Graal.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.common/src/org/graalvm/compiler/core/common/util/ModuleAPI.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/CheckGraalInvariants.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/InterfaceMethodHandleTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/OptionsVerifierTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/StaticInterfaceFieldTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/UnbalancedMonitorsTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/VerifyBailoutUsageTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/VerifyDebugUsageTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/VerifyVirtualizableTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/debug/VerifyMethodMetricsTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/tutorial/StaticAnalysis.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/CheckGraalIntrinsics.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/ConstantPoolSubstitutionsTests.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/TestIntrinsicCompiles.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/CompileTheWorld.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/HotSpotGraalCompilerFactory.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/HotSpotGraalJVMCIServiceLocator.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotGraphBuilderPlugins.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/meta/HotSpotInvocationPlugins.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/AESCryptSubstitutions.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/CRC32Substitutions.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/ThreadSubstitutions.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/graphbuilderconf/InvocationPlugin.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.nodes/src/org/graalvm/compiler/nodes/graphbuilderconf/InvocationPlugins.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.printer/src/org/graalvm/compiler/printer/GraphPrinter.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.test/src/org/graalvm/compiler/replacements/test/DeoptimizeOnExceptionTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.serviceprovider/src/org/graalvm/compiler/serviceprovider/GraalServices.java + src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.serviceprovider/src/org/graalvm/compiler/serviceprovider/JDK9Method.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.test/src/org/graalvm/compiler/test/JLModule.java ! src/share/vm/runtime/arguments.cpp ! test/compiler/jvmci/JVM_GetJVMCIRuntimeTest.java ! test/compiler/jvmci/TestJVMCIPrintProperties.java ! test/compiler/jvmci/compilerToVM/AsResolvedJavaMethodTest.java ! test/compiler/jvmci/compilerToVM/DoNotInlineOrCompileTest.java ! test/compiler/jvmci/compilerToVM/FindUniqueConcreteMethodTest.java ! test/compiler/jvmci/compilerToVM/GetBytecodeTest.java ! test/compiler/jvmci/compilerToVM/GetClassInitializerTest.java ! test/compiler/jvmci/compilerToVM/GetConstantPoolTest.java ! test/compiler/jvmci/compilerToVM/GetExceptionTableTest.java ! test/compiler/jvmci/compilerToVM/GetImplementorTest.java ! test/compiler/jvmci/compilerToVM/GetLineNumberTableTest.java ! test/compiler/jvmci/compilerToVM/GetLocalVariableTableTest.java ! test/compiler/jvmci/compilerToVM/GetNextStackFrameTest.java ! test/compiler/jvmci/compilerToVM/GetResolvedJavaMethodTest.java ! test/compiler/jvmci/compilerToVM/GetResolvedJavaTypeTest.java ! test/compiler/jvmci/compilerToVM/GetStackTraceElementTest.java ! test/compiler/jvmci/compilerToVM/GetSymbolTest.java ! test/compiler/jvmci/compilerToVM/GetVtableIndexForInterfaceTest.java ! test/compiler/jvmci/compilerToVM/HasCompiledCodeForOSRTest.java ! test/compiler/jvmci/compilerToVM/HasFinalizableSubclassTest.java ! test/compiler/jvmci/compilerToVM/HasNeverInlineDirectiveTest.java ! test/compiler/jvmci/compilerToVM/InvalidateInstalledCodeTest.java ! test/compiler/jvmci/compilerToVM/IsCompilableTest.java ! test/compiler/jvmci/compilerToVM/IsMatureVsReprofileTest.java ! test/compiler/jvmci/compilerToVM/LookupKlassInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupKlassRefIndexInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupMethodInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupNameAndTypeRefIndexInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupNameInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupSignatureInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupTypeTest.java ! test/compiler/jvmci/compilerToVM/MaterializeVirtualObjectTest.java ! test/compiler/jvmci/compilerToVM/MethodIsIgnoredBySecurityStackWalkTest.java ! test/compiler/jvmci/compilerToVM/ReadConfigurationTest.java ! test/compiler/jvmci/compilerToVM/ReprofileTest.java ! test/compiler/jvmci/compilerToVM/ResolveConstantInPoolTest.java ! test/compiler/jvmci/compilerToVM/ResolveFieldInPoolTest.java ! test/compiler/jvmci/compilerToVM/ResolveMethodTest.java ! test/compiler/jvmci/compilerToVM/ResolvePossiblyCachedConstantInPoolTest.java ! test/compiler/jvmci/compilerToVM/ResolveTypeInPoolTest.java ! test/compiler/jvmci/compilerToVM/ShouldInlineMethodTest.java ! test/compiler/jvmci/errors/TestInvalidCompilationResult.java ! test/compiler/jvmci/errors/TestInvalidDebugInfo.java ! test/compiler/jvmci/errors/TestInvalidOopMap.java ! test/compiler/jvmci/events/JvmciNotifyInstallEventTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/DataPatchTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/InterpreterFrameSizeTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/MaxOopMapStackOffsetTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/SimpleCodeInstallationTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/SimpleDebugInfoTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/VirtualObjectDebugInfoTest.java ! test/compiler/jvmci/jdk.vm.ci.hotspot.test/src/jdk/vm/ci/hotspot/test/HotSpotConstantReflectionProviderTest.java ! test/compiler/jvmci/jdk.vm.ci.hotspot.test/src/jdk/vm/ci/hotspot/test/MemoryAccessProviderTest.java ! test/compiler/jvmci/jdk.vm.ci.hotspot.test/src/jdk/vm/ci/hotspot/test/MethodHandleAccessProviderTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ConstantTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/RedefineClassTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ResolvedJavaTypeResolveConcreteMethodTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ResolvedJavaTypeResolveMethodTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestConstantReflectionProvider.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaField.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaMethod.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaType.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestMetaAccessProvider.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaField.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaMethod.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java ! test/compiler/jvmci/meta/StableFieldTest.java Changeset: a0bd5a8c4e11 Author: alanb Date: 2017-05-04 07:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/a0bd5a8c4e11 8178380: Module system implementation refresh (5/2017) Reviewed-by: lfoltan, hseigel, mchung, sspitsyn Contributed-by: alan.bateman at oracle.com, serguei.spitsyn at oracle.com ! src/share/vm/classfile/moduleEntry.cpp ! src/share/vm/classfile/modules.cpp ! src/share/vm/classfile/packageEntry.cpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/logging/logTag.hpp ! src/share/vm/oops/instanceKlass.cpp ! src/share/vm/prims/jvmti.xml ! src/share/vm/prims/jvmtiExport.cpp ! src/share/vm/prims/jvmtiExport.hpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/globals.hpp ! src/share/vm/runtime/thread.cpp ! src/share/vm/services/attachListener.cpp ! src/share/vm/services/diagnosticCommand.cpp ! test/compiler/jsr292/RedefineMethodUsedByMultipleMethodHandles.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/RedefineClassTest.java ! test/compiler/profiling/spectrapredefineclass/Launcher.java ! test/compiler/profiling/spectrapredefineclass_classloaders/Launcher.java ! test/runtime/CommandLine/OptionsValidation/TestOptionsWithRangesDynamic.java ! test/runtime/Metaspace/DefineClass.java ! test/runtime/logging/ModulesTest.java ! test/runtime/logging/StartupTimeTest.java ! test/runtime/modules/JVMAddModuleExports.java ! test/runtime/modules/JVMAddModulePackage.java ! test/runtime/modules/JVMDefineModule.java ! test/runtime/modules/ModuleStress/ModuleStress.java ! test/runtime/modules/ModuleStress/ModuleStressGC.java Changeset: fa79b980fa63 Author: alanb Date: 2017-05-04 09:43 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/fa79b980fa63 Merge ! src/share/vm/runtime/arguments.cpp ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/RedefineClassTest.java Changeset: 9906107f9050 Author: alanb Date: 2017-05-04 11:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/9906107f9050 Merge ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.test/src/org/graalvm/compiler/test/JLModule.java ! src/share/vm/classfile/moduleEntry.cpp ! src/share/vm/classfile/modules.cpp ! src/share/vm/classfile/packageEntry.cpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/logging/logTag.hpp ! src/share/vm/oops/instanceKlass.cpp ! src/share/vm/prims/jvmti.xml ! src/share/vm/prims/jvmtiExport.cpp ! src/share/vm/prims/jvmtiExport.hpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/globals.hpp ! src/share/vm/runtime/thread.cpp ! src/share/vm/services/attachListener.cpp ! test/compiler/jvmci/JVM_GetJVMCIRuntimeTest.java ! test/compiler/jvmci/compilerToVM/AsResolvedJavaMethodTest.java ! test/compiler/jvmci/compilerToVM/DoNotInlineOrCompileTest.java ! test/compiler/jvmci/compilerToVM/FindUniqueConcreteMethodTest.java ! test/compiler/jvmci/compilerToVM/GetBytecodeTest.java ! test/compiler/jvmci/compilerToVM/GetClassInitializerTest.java ! test/compiler/jvmci/compilerToVM/GetConstantPoolTest.java ! test/compiler/jvmci/compilerToVM/GetExceptionTableTest.java ! test/compiler/jvmci/compilerToVM/GetImplementorTest.java ! test/compiler/jvmci/compilerToVM/GetLineNumberTableTest.java ! test/compiler/jvmci/compilerToVM/GetLocalVariableTableTest.java ! test/compiler/jvmci/compilerToVM/GetNextStackFrameTest.java ! test/compiler/jvmci/compilerToVM/GetResolvedJavaMethodTest.java ! test/compiler/jvmci/compilerToVM/GetResolvedJavaTypeTest.java ! test/compiler/jvmci/compilerToVM/GetStackTraceElementTest.java ! test/compiler/jvmci/compilerToVM/GetSymbolTest.java ! test/compiler/jvmci/compilerToVM/GetVtableIndexForInterfaceTest.java ! test/compiler/jvmci/compilerToVM/HasCompiledCodeForOSRTest.java ! test/compiler/jvmci/compilerToVM/HasFinalizableSubclassTest.java ! test/compiler/jvmci/compilerToVM/HasNeverInlineDirectiveTest.java ! test/compiler/jvmci/compilerToVM/LookupKlassInPoolTest.java ! test/compiler/jvmci/compilerToVM/LookupTypeTest.java ! test/compiler/jvmci/compilerToVM/MethodIsIgnoredBySecurityStackWalkTest.java ! test/compiler/jvmci/compilerToVM/ReprofileTest.java ! test/compiler/jvmci/compilerToVM/ResolveConstantInPoolTest.java ! test/compiler/jvmci/compilerToVM/ResolveMethodTest.java ! test/compiler/jvmci/compilerToVM/ResolveTypeInPoolTest.java ! test/compiler/jvmci/compilerToVM/ShouldInlineMethodTest.java ! test/compiler/jvmci/errors/TestInvalidCompilationResult.java ! test/compiler/jvmci/errors/TestInvalidDebugInfo.java ! test/compiler/jvmci/errors/TestInvalidOopMap.java ! test/compiler/jvmci/events/JvmciNotifyInstallEventTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/DataPatchTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/SimpleCodeInstallationTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/SimpleDebugInfoTest.java ! test/compiler/jvmci/jdk.vm.ci.code.test/src/jdk/vm/ci/code/test/VirtualObjectDebugInfoTest.java ! test/compiler/jvmci/jdk.vm.ci.hotspot.test/src/jdk/vm/ci/hotspot/test/MethodHandleAccessProviderTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ConstantTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/RedefineClassTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ResolvedJavaTypeResolveConcreteMethodTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ResolvedJavaTypeResolveMethodTest.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestConstantReflectionProvider.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaField.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaMethod.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestJavaType.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestMetaAccessProvider.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaField.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaMethod.java ! test/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java ! test/runtime/logging/ModulesTest.java ! test/runtime/modules/JVMAddModuleExports.java ! test/runtime/modules/JVMAddModulePackage.java ! test/runtime/modules/JVMDefineModule.java ! test/runtime/modules/ModuleStress/ModuleStressGC.java From alan.bateman at oracle.com Thu May 4 10:54:27 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 10:54:27 +0000 Subject: hg: jigsaw/jake/jdk: 34 new changesets Message-ID: <201705041054.v44AsSMZ028520@aojmv0008.oracle.com> Changeset: fa4bdd8f0eed Author: chegar Date: 2017-04-25 14:46 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/fa4bdd8f0eed 8158270: MulticastSendReceiveTests.java failed with "Expected message not received" Reviewed-by: alanb ! test/java/nio/channels/DatagramChannel/NetworkConfiguration.java Changeset: 10eb52093bb8 Author: mchung Date: 2017-04-25 09:09 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/10eb52093bb8 8179025: Exclude deployment modules from FieldSetAccessibleTest.java and VerifyJimage.java Reviewed-by: alanb ! test/ProblemList.txt ! test/java/lang/Class/getDeclaredField/FieldSetAccessibleTest.java ! test/jdk/modules/etc/JdkQualifiedExportTest.java ! test/tools/jimage/VerifyJimage.java Changeset: fd5c621ad3b1 Author: ksrini Date: 2017-04-25 07:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/fd5c621ad3b1 8178725: provide way to link to external documentation Reviewed-by: erikj, ihse, jjg + make/src/classes/build/tools/taglet/ExtLink.java Changeset: 1c6407c57773 Author: ihse Date: 2017-04-26 14:35 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/1c6407c57773 8178042: Allow custom taglets Reviewed-by: erikj, mchung ! make/CompileTools.gmk Changeset: 7c04ab31b4d6 Author: naoto Date: 2017-04-26 09:37 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7c04ab31b4d6 8178412: Doc link updates for i18n Reviewed-by: ksrini ! src/java.desktop/share/classes/java/awt/im/package-info.java ! src/java.desktop/share/classes/java/awt/im/spi/package-info.java ! src/java.desktop/share/classes/javax/swing/colorchooser/package-info.java ! src/java.desktop/share/classes/javax/swing/text/JTextComponent.java Changeset: bd0e423e5756 Author: jjg Date: 2017-04-26 13:38 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/bd0e423e5756 8179364: update " characters Reviewed-by: ksrini, mullan ! src/java.base/share/classes/javax/crypto/CipherInputStream.java ! src/java.base/share/classes/javax/crypto/CryptoPermission.java ! src/java.base/share/classes/javax/crypto/CryptoPolicyParser.java ! src/java.xml.crypto/share/classes/javax/xml/crypto/dsig/XMLSignContext.java ! src/java.xml.crypto/share/classes/javax/xml/crypto/dsig/XMLValidateContext.java Changeset: e28156f27e0a Author: ihse Date: 2017-05-03 08:46 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e28156f27e0a 8179453: Add a proper SetupProcessMarkdown Reviewed-by: erikj + make/data/docs-resources/specs/resources/jdk-default.css Changeset: 0fa2b22476cd Author: ksrini Date: 2017-05-03 07:39 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/0fa2b22476cd 8179538: Update jdk.jdi to be HTML-5 friendly Reviewed-by: mchung, sspitsyn ! src/jdk.jdi/share/classes/com/sun/jdi/ClassType.java ! src/jdk.jdi/share/classes/com/sun/jdi/InterfaceType.java ! src/jdk.jdi/share/classes/com/sun/jdi/JDIPermission.java ! src/jdk.jdi/share/classes/com/sun/jdi/Location.java ! src/jdk.jdi/share/classes/com/sun/jdi/ObjectReference.java ! src/jdk.jdi/share/classes/com/sun/jdi/Type.java ! src/jdk.jdi/share/classes/com/sun/jdi/Value.java ! src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachineManager.java ! src/jdk.jdi/share/classes/com/sun/jdi/doc-files/signature.html ! src/jdk.jdi/share/classes/module-info.java Changeset: a50b619ac159 Author: lancea Date: 2017-05-03 12:58 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/a50b619ac159 8078267: Add test to verify that a module based JDBC driver via the service-provider loading mechanism Reviewed-by: joehw + test/java/sql/driverModuleTests/DriverManagerModuleTests.java + test/java/sql/modules/luckydogdriver/luckydogtennis/LuckyDogDriver.java + test/java/sql/modules/luckydogdriver/luckydogtennis/StubConnection.java + test/java/sql/modules/luckydogdriver/module-info.java + test/java/sql/modules/mystubdriver/com/luckydogtennis/StubConnection.java + test/java/sql/modules/mystubdriver/com/luckydogtennis/StubDriver.java + test/java/sql/modules/mystubdriver/module-info.java Changeset: 797880157074 Author: mli Date: 2017-05-03 21:15 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/797880157074 8179451: Confidential copyright header in openjdk Summary: Using Oracle OpenJDK copyright notices Reviewed-by: weijun Contributed-by: John Jiang ! test/sun/security/ssl/CertPathRestrictions/JSSEClient.java ! test/sun/security/ssl/CertPathRestrictions/JSSEServer.java ! test/sun/security/ssl/CertPathRestrictions/TLSRestrictions.java Changeset: 73113c19a5df Author: alanb Date: 2017-05-04 07:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/73113c19a5df 8178380: Module system implementation refresh (5/2017) Reviewed-by: mchung, alanb, sspitsyn Contributed-by: alan.bateman at oracle.com, mandy.chung at oracle.com, chris.hegarty at oracle.com ! make/mapfiles/libinstrument/mapfile-vers ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/classes/java/lang/Module.java ! src/java.base/share/classes/java/lang/SecurityManager.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/java/lang/module/Configuration.java ! src/java.base/share/classes/java/lang/module/ModuleDescriptor.java ! src/java.base/share/classes/java/lang/module/ModuleFinder.java ! src/java.base/share/classes/java/lang/module/Resolver.java ! src/java.base/share/classes/java/nio/file/FileSystems.java ! src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java ! src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java ! src/java.base/share/classes/jdk/internal/module/Checks.java ! src/java.base/share/classes/jdk/internal/module/ClassFileAttributes.java ! src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java ! src/java.base/share/classes/jdk/internal/module/ModuleHashesBuilder.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfo.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfoExtender.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfoWriter.java ! src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java ! src/java.base/share/classes/jdk/internal/module/ModulePatcher.java ! src/java.base/share/classes/jdk/internal/module/ModulePath.java ! src/java.base/share/classes/jdk/internal/module/ModuleReferences.java ! src/java.base/share/classes/jdk/internal/module/ModuleResolution.java ! src/java.base/share/classes/jdk/internal/module/ModuleTarget.java ! src/java.base/share/classes/jdk/internal/module/Modules.java ! src/java.base/share/classes/jdk/internal/module/Resources.java ! src/java.base/share/classes/module-info.java ! src/java.base/share/classes/sun/launcher/LauncherHelper.java ! src/java.base/share/classes/sun/launcher/resources/launcher.properties ! src/java.base/share/native/libjli/emessages.h ! src/java.base/share/native/libjli/java.c ! src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java ! src/java.instrument/share/classes/java/lang/instrument/package.html ! src/java.instrument/share/classes/module-info.java ! src/java.instrument/share/classes/sun/instrument/InstrumentationImpl.java ! src/java.instrument/share/native/libinstrument/InstrumentationImplNativeMethods.c ! src/java.instrument/share/native/libinstrument/InvocationAdapter.c ! src/java.management/share/classes/java/lang/management/ThreadInfo.java ! src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java ! src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java ! src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java ! src/jdk.attach/share/classes/sun/tools/attach/HotSpotVirtualMachine.java ! src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java ! src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java ! src/jdk.jartool/share/classes/sun/tools/jar/Main.java ! src/jdk.jartool/share/classes/sun/tools/jar/Validator.java ! src/jdk.jartool/share/classes/sun/tools/jar/resources/jar.properties ! src/jdk.jlink/share/classes/jdk/tools/jlink/builder/DefaultImageBuilder.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/ImagePluginStack.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/Platform.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/ResourcePoolManager.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/ReleaseInfoPlugin.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/SystemModulesPlugin.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/plugin/ResourcePoolModule.java ! src/jdk.jlink/share/classes/jdk/tools/jmod/JmodTask.java ! src/jdk.jlink/share/classes/jdk/tools/jmod/resources/jmod.properties ! test/com/sun/management/HotSpotDiagnosticMXBean/CheckOrigin.java + test/com/sun/tools/attach/AttachSelf.java ! test/com/sun/tools/attach/RunnerUtil.java ! test/com/sun/tools/attach/java.policy.allow + test/com/sun/tools/attach/modules/Agent.java + test/com/sun/tools/attach/modules/Driver.java + test/com/sun/tools/attach/modules/m/module-info.java + test/com/sun/tools/attach/modules/m/p/Main.java + test/java/lang/instrument/executableJAR/Agent.java + test/java/lang/instrument/executableJAR/AgentHelper.java + test/java/lang/instrument/executableJAR/ExecJarWithAgent.java + test/java/lang/instrument/executableJAR/Main.java ! test/java/lang/invoke/DefineClassTest.java ! test/java/lang/module/AutomaticModulesTest.java ! test/java/lang/module/ConfigurationTest.java ! test/java/lang/module/ModuleDescriptorTest.java + test/java/lang/module/customfs/ModulesInCustomFileSystem.java + test/java/lang/module/customfs/m1/module-info.java + test/java/lang/module/customfs/m1/p/Main.java + test/java/lang/module/customfs/m2/module-info.java + test/java/lang/module/customfs/m2/q/Hello.java ! test/java/nio/file/spi/SetDefaultProvider.java ! test/java/nio/file/spi/TestProvider.java + test/java/nio/file/spi/m/module-info.java + test/java/nio/file/spi/m/p/Main.java ! test/java/util/logging/TestLoggerWeakRefLeak.java ! test/lib/testlibrary/ModuleTargetHelper.java ! test/sun/management/jmxremote/bootstrap/JvmstatCountersTest.java ! test/tools/jar/mmrjar/Basic.java ! test/tools/jar/modularJar/Basic.java + test/tools/jar/multiRelease/whitebox/Driver.java + test/tools/jar/multiRelease/whitebox/jdk.jartool/sun/tools/jar/ValidatorComparatorTest.java ! test/tools/jlink/IntegrationTest.java ! test/tools/jlink/JLinkNegativeTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/SystemModulesTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/UserModuleTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/src/m1/p1/Main.java ! test/tools/jlink/plugins/SystemModuleDescriptors/src/m4/p4/Main.java ! test/tools/jmod/JmodTest.java + test/tools/launcher/modules/describe/DescribeModuleTest.java ! test/tools/launcher/modules/listmods/ListModsTest.java - test/tools/launcher/modules/listmods/src/java.transaction/javax/transaction/atomic/Atomic.java ! test/tools/launcher/modules/listmods/src/java.transaction/module-info.java ! test/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java + test/tools/launcher/modules/showmoduleresolution/ShowModuleResolutionTest.java + test/tools/launcher/modules/validate/ValidateModulesTest.java ! test/tools/pack200/pack200-verifier/src/xmlkit/ClassReader.java Changeset: ab86e2c522db Author: alanb Date: 2017-05-04 09:43 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/ab86e2c522db Merge ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.management/share/classes/java/lang/management/ThreadInfo.java - test/tools/launcher/modules/listmods/src/java.transaction/javax/transaction/atomic/Atomic.java Changeset: 30e6e822e0b7 Author: alanb Date: 2017-05-04 11:00 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/30e6e822e0b7 Merge ! make/gensrc/GensrcCharsetCoder.gmk ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/classes/java/lang/Module.java ! src/java.base/share/classes/java/lang/SecurityManager.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/java/lang/module/Configuration.java ! src/java.base/share/classes/java/lang/module/ModuleDescriptor.java ! src/java.base/share/classes/java/lang/module/ModuleFinder.java ! src/java.base/share/classes/java/lang/module/Resolver.java ! src/java.base/share/classes/java/lang/reflect/Proxy.java ! src/java.base/share/classes/java/nio/charset/Charset.java ! src/java.base/share/classes/java/nio/file/FileSystems.java ! src/java.base/share/classes/java/util/ResourceBundle.java ! src/java.base/share/classes/java/util/ServiceLoader.java ! src/java.base/share/classes/jdk/internal/loader/BuiltinClassLoader.java ! src/java.base/share/classes/jdk/internal/misc/JavaLangAccess.java ! src/java.base/share/classes/jdk/internal/misc/VM.java ! src/java.base/share/classes/jdk/internal/module/Checks.java ! src/java.base/share/classes/jdk/internal/module/ClassFileAttributes.java ! src/java.base/share/classes/jdk/internal/module/ModuleBootstrap.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfo.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfoExtender.java ! src/java.base/share/classes/jdk/internal/module/ModuleInfoWriter.java ! src/java.base/share/classes/jdk/internal/module/ModuleLoaderMap.java ! src/java.base/share/classes/jdk/internal/module/ModulePatcher.java ! src/java.base/share/classes/jdk/internal/module/ModulePath.java ! src/java.base/share/classes/jdk/internal/module/ModuleReferences.java ! src/java.base/share/classes/jdk/internal/module/ModuleResolution.java ! src/java.base/share/classes/jdk/internal/module/ModuleTarget.java ! src/java.base/share/classes/jdk/internal/module/Modules.java ! src/java.base/share/classes/jdk/internal/module/Resources.java ! src/java.base/share/classes/module-info.java ! src/java.base/share/classes/sun/launcher/LauncherHelper.java ! src/java.base/share/classes/sun/launcher/resources/launcher.properties ! src/java.base/share/native/libjli/emessages.h ! src/java.base/share/native/libjli/java.c ! src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java ! src/java.instrument/share/classes/java/lang/instrument/package.html ! src/java.instrument/share/classes/module-info.java ! src/java.instrument/share/classes/sun/instrument/InstrumentationImpl.java ! src/java.instrument/share/native/libinstrument/InvocationAdapter.c ! src/jdk.attach/share/classes/sun/tools/attach/HotSpotVirtualMachine.java ! src/jdk.jartool/share/classes/sun/tools/jar/Main.java ! src/jdk.jartool/share/classes/sun/tools/jar/resources/jar.properties ! src/jdk.jdi/share/classes/module-info.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/builder/DefaultImageBuilder.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/ImagePluginStack.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/Platform.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/ResourcePoolManager.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/ReleaseInfoPlugin.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/SystemModulesPlugin.java ! src/jdk.jlink/share/classes/jdk/tools/jmod/JmodTask.java ! src/jdk.jlink/share/classes/jdk/tools/jmod/resources/jmod.properties ! src/jdk.security.auth/share/classes/com/sun/security/auth/module/LdapLoginModule.java ! test/ProblemList.txt ! test/TEST.groups ! test/com/sun/management/HotSpotDiagnosticMXBean/CheckOrigin.java ! test/com/sun/tools/attach/AttachSelf.java ! test/com/sun/tools/attach/modules/m/p/Main.java - test/demo/jvmti/Context.java - test/demo/jvmti/DemoRun.java - test/demo/jvmti/HeapUser.java - test/demo/jvmti/Hello.java - test/demo/jvmti/compiledMethodLoad/CompiledMethodLoadTest.java - test/demo/jvmti/gctest/BigHello.java - test/demo/jvmti/gctest/Gctest.java - test/demo/jvmti/heapTracker/HeapTrackerTest.java - test/demo/jvmti/heapViewer/HeapViewerTest.java - test/demo/jvmti/minst/MinstExample.java - test/demo/jvmti/minst/MinstTest.java - test/demo/jvmti/versionCheck/FailsWhenJvmtiVersionDiffers.java - test/demo/jvmti/waiters/WaitersTest.java ! test/java/lang/Class/getDeclaredField/FieldSetAccessibleTest.java ! test/java/lang/instrument/executableJAR/ExecJarWithAgent.java ! test/java/lang/invoke/DefineClassTest.java ! test/java/lang/module/AutomaticModulesTest.java ! test/java/lang/module/ConfigurationTest.java ! test/java/lang/module/ModuleDescriptorTest.java ! test/jdk/modules/incubator/ImageModules.java ! test/sun/management/jmxremote/bootstrap/JvmstatCountersTest.java ! test/tools/jar/mmrjar/Basic.java ! test/tools/jar/modularJar/Basic.java ! test/tools/jimage/VerifyJimage.java ! test/tools/jlink/IntegrationTest.java ! test/tools/jlink/JLinkNegativeTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/SystemModulesTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/UserModuleTest.java ! test/tools/jlink/plugins/SystemModuleDescriptors/src/m1/p1/Main.java ! test/tools/jmod/JmodTest.java ! test/tools/launcher/modules/listmods/ListModsTest.java ! test/tools/launcher/modules/listmods/src/java.transaction/module-info.java ! test/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java ! test/tools/pack200/pack200-verifier/src/xmlkit/ClassReader.java From greggwon at cox.net Thu May 4 15:29:14 2017 From: greggwon at cox.net (Gregg Wonderly) Date: Thu, 4 May 2017 10:29:14 -0500 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: References: <5907D11D.20609@oracle.com> <5908F8A9.1000205@oracle.com> Message-ID: <8BB9D503-D6BC-4948-A73F-D22FF5A6835E@cox.net> If there is not already such an exception, it would seem like a good idea to have an exception that formats such a message from constructor parameters providing the details so that it?s the same everywhere, and so that it can be changed in once place if needed. Gregg > On May 3, 2017, at 9:48 AM, Kevin Rushforth wrote: > > OK, I'll create a more informative message. I think it will be more clear in the message to just say that it needs to "open" the package to javafx.base, since that would be the recommendation for a package that isn't already exported unconditionally. I'll send out a new webrev this morning with all feedback incorporated. > > -- Kevin > > > Mandy Chung wrote: >>> On May 2, 2017, at 2:22 PM, Kevin Rushforth wrote: >>> >>> Here is the message: >>> >>> IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base >>> >> >> It would be better to emit a more informative message e.g. ?javafx.base cannot access class com.foo (in module foo.app). Either exports com.foo unqualifiedly or open com.foo to javafx.base?. Also in MethodUtil::invoke >> 61 if (!module.isExported(packageName)) { >> You can do this check only if module.isNamed. >> >> >>> It is roughly the same message that any similar illegal access would generate (e.g., we get similar error messages when FXML tries to call setAccessible for a field annotated with @FXML if the module is not "open" to javafx.fxml). >>> >>> >>>> javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java >>>> javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java >>>> javafx.web/src/main/java/com/sun/webkit/MethodHelper.java >>>> 45 public static Object invoke(Method m, Object obj, Object[] params) >>>> >>>> To avoid 3 ModuleHelper classes, the invoke method can take >>>> the callerModule argument to replace this line: 56 final Module thisModule = MethodHelper.class.getModule(); >>>> >>> I'm fairly certain that won't work. Module::addOpens is caller sensitive and will only work when called from the module in question. >>> >>> >> >> You are right. Wont?t work. >> >>>> javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java >>>> There are a few other public methods which I think JavaFX doesn?t >>>> need and can be removed. >>>> >>> Yes, I could do this to reduce the public footprint of the class. To minimize the diffs between the original and our copy, I might just comment out the "public". That would also make it easier to add them back in a future version (e.g., to eventually get rid of all dependency on sun.reflect.misc). Thoughts? >>> >> >> I will leave it up to you. >> Mandy >> >> From kevin.rushforth at oracle.com Thu May 4 15:37:52 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Thu, 04 May 2017 08:37:52 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <8BB9D503-D6BC-4948-A73F-D22FF5A6835E@cox.net> References: <5907D11D.20609@oracle.com> <5908F8A9.1000205@oracle.com> <8BB9D503-D6BC-4948-A73F-D22FF5A6835E@cox.net> Message-ID: <590B4AD0.4050705@oracle.com> This seems like something that could be useful, although at this point in the release we would more likely do it for JDK 10. I do note that including the class that made the illegal access is generally a good idea when that class is attempting the access on its own behalf. For frameworks such as JavaFX beans or FXML, we are making the request on behalf of an application, it wasn't as helpful to have the class itself be in the error message (which was Mandy's main point). So there are at least two cases to consider if we end up creating such a utility method. Thanks. -- Kevin Gregg Wonderly wrote: > If there is not already such an exception, it would seem like a good idea to have an exception that formats such a message from constructor parameters providing the details so that it?s the same everywhere, and so that it can be changed in once place if needed. > > Gregg > > >> On May 3, 2017, at 9:48 AM, Kevin Rushforth wrote: >> >> OK, I'll create a more informative message. I think it will be more clear in the message to just say that it needs to "open" the package to javafx.base, since that would be the recommendation for a package that isn't already exported unconditionally. I'll send out a new webrev this morning with all feedback incorporated. >> >> -- Kevin >> >> >> Mandy Chung wrote: >> >>>> On May 2, 2017, at 2:22 PM, Kevin Rushforth wrote: >>>> >>>> Here is the message: >>>> >>>> IllegalAccessException: class com.sun.javafx.property.MethodHelper cannot access class com.foo (in module foo.app) because module foo.app does not open com.foo to javafx.base >>>> >>>> >>> It would be better to emit a more informative message e.g. ?javafx.base cannot access class com.foo (in module foo.app). Either exports com.foo unqualifiedly or open com.foo to javafx.base?. Also in MethodUtil::invoke >>> 61 if (!module.isExported(packageName)) { >>> You can do this check only if module.isNamed. >>> >>> >>> >>>> It is roughly the same message that any similar illegal access would generate (e.g., we get similar error messages when FXML tries to call setAccessible for a field annotated with @FXML if the module is not "open" to javafx.fxml). >>>> >>>> >>>> >>>>> javafx.base/src/main/java/com/sun/javafx/property/MethodHelper.java >>>>> javafx.fxml/src/main/java/com/sun/javafx/fxml/MethodHelper.java >>>>> javafx.web/src/main/java/com/sun/webkit/MethodHelper.java >>>>> 45 public static Object invoke(Method m, Object obj, Object[] params) >>>>> >>>>> To avoid 3 ModuleHelper classes, the invoke method can take >>>>> the callerModule argument to replace this line: 56 final Module thisModule = MethodHelper.class.getModule(); >>>>> >>>>> >>>> I'm fairly certain that won't work. Module::addOpens is caller sensitive and will only work when called from the module in question. >>>> >>>> >>>> >>> You are right. Wont?t work. >>> >>> >>>>> javafx.base/src/main/java/com/sun/javafx/reflect/MethodUtil.java >>>>> There are a few other public methods which I think JavaFX doesn?t >>>>> need and can be removed. >>>>> >>>>> >>>> Yes, I could do this to reduce the public footprint of the class. To minimize the diffs between the original and our copy, I might just comment out the "public". That would also make it easier to add them back in a future version (e.g., to eventually get rid of all dependency on sun.reflect.misc). Thoughts? >>>> >>>> >>> I will leave it up to you. >>> Mandy >>> >>> >>> > > From mark.reinhold at oracle.com Thu May 4 17:39:06 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 04 May 2017 10:39:06 -0700 Subject: Revised proposal for #AutomaticModuleNames Message-ID: <20170504103906.689386446@eggemoggin.niobe.net> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and Robert Scholte, for the extensive feedback. http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html TL;DR: Keep automatic modules, bring back the module-name JAR-file manifest attribute, and strongly recommend reverse-DNS module names. Comments? - Mark From alan.bateman at oracle.com Thu May 4 18:19:59 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:19:59 +0000 Subject: hg: jigsaw/jake/corba: 2 new changesets Message-ID: <201705041819.v44IJxur004212@aojmv0008.oracle.com> Changeset: b2218d41edef Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/b2218d41edef Added tag jdk-9+168 for changeset 03a2cc9c8a1e ! .hgtags Changeset: c8f59dfe1269 Author: alanb Date: 2017-05-04 19:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/c8f59dfe1269 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:00 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:00 +0000 Subject: hg: jigsaw/jake/jaxws: 2 new changesets Message-ID: <201705041820.v44IK08P004338@aojmv0008.oracle.com> Changeset: e66ea56c8751 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/e66ea56c8751 Added tag jdk-9+168 for changeset 2746716dcc5a ! .hgtags Changeset: 5a7209da3317 Author: alanb Date: 2017-05-04 19:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/5a7209da3317 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:19:58 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:19:58 +0000 Subject: hg: jigsaw/jake: 2 new changesets Message-ID: <201705041819.v44IJwND004178@aojmv0008.oracle.com> Changeset: 8e1d0bed44e5 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/8e1d0bed44e5 Added tag jdk-9+168 for changeset 143d4c87bc1e ! .hgtags Changeset: c6f2f8eaafb2 Author: alanb Date: 2017-05-04 19:01 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/c6f2f8eaafb2 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:00 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:00 +0000 Subject: hg: jigsaw/jake/jaxp: 2 new changesets Message-ID: <201705041820.v44IK0Nl004262@aojmv0008.oracle.com> Changeset: df63b2d22ac6 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/df63b2d22ac6 Added tag jdk-9+168 for changeset 23a87f409371 ! .hgtags Changeset: a89495528083 Author: alanb Date: 2017-05-04 19:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/a89495528083 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:03 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:03 +0000 Subject: hg: jigsaw/jake/nashorn: 2 new changesets Message-ID: <201705041820.v44IK3b0004399@aojmv0008.oracle.com> Changeset: c78163ad1a4e Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/c78163ad1a4e Added tag jdk-9+168 for changeset 0f81cde5a1f7 ! .hgtags Changeset: 61cbf87f8b49 Author: alanb Date: 2017-05-04 19:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/61cbf87f8b49 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:02 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:02 +0000 Subject: hg: jigsaw/jake/langtools: 2 new changesets Message-ID: <201705041820.v44IK2bi004394@aojmv0008.oracle.com> Changeset: c61dc27c48d1 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/c61dc27c48d1 Added tag jdk-9+168 for changeset bc21e5ba6bf1 ! .hgtags Changeset: 3ed3d69a8cc5 Author: alanb Date: 2017-05-04 19:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/3ed3d69a8cc5 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:04 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:04 +0000 Subject: hg: jigsaw/jake/jdk: 2 new changesets Message-ID: <201705041820.v44IK4UU004434@aojmv0008.oracle.com> Changeset: 8bfab0e73164 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/8bfab0e73164 Added tag jdk-9+168 for changeset e78da9db6299 ! .hgtags Changeset: 052d759c36a8 Author: alanb Date: 2017-05-04 19:02 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/052d759c36a8 Merge ! .hgtags From alan.bateman at oracle.com Thu May 4 18:20:04 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Thu, 04 May 2017 18:20:04 +0000 Subject: hg: jigsaw/jake/hotspot: 2 new changesets Message-ID: <201705041820.v44IK4Fe004440@aojmv0008.oracle.com> Changeset: f36e864e66a7 Author: lana Date: 2017-05-04 16:38 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/f36e864e66a7 Added tag jdk-9+168 for changeset fbb9c8026495 ! .hgtags Changeset: 1d55ec6b9bc7 Author: alanb Date: 2017-05-04 19:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/1d55ec6b9bc7 Merge ! .hgtags From markus_keller at ch.ibm.com Thu May 4 18:51:03 2017 From: markus_keller at ch.ibm.com (Markus Keller) Date: Thu, 4 May 2017 20:51:03 +0200 Subject: Views on JSR 376 from the Eclipse JDT team Message-ID: Views on JSR 376 from the Eclipse JDT team ====================================== Here's a high-level feedback on things the Eclipse JDT team is missing in the Java Platform Module System spec (JSR 376). (This message is based on an internal note from Markus Keller but basically reflects the position of the whole team) The first problem is that's it's unclear what should be the spec. http://openjdk.java.net/projects/jigsaw/spec/ says "This is the primary web page for JSR 376", but the JLS9 & JVMS9 drafts linked there are dated 2017-02-22. Lots of discussions have been going on since then, and we can't be expected to pick out informal answers buried in mailing list posts. => Without a complete and up-to-date spec, the only reasonable vote on JSR 376 is "no". The other big thing is that the JLS draft specifies the syntax of the module-info.java file, but it is quite vague about the semantics of modules. JLS9 12 "Execution" explains how class loading is supposed to work in the JVM, but it's unclear how modules and and their access restrictions should come into the picture here. Layers are not even mentioned anywhere. Before Java 9, classloading and discovery of .class files was only a run-time concern. During compilation, the assumption was that all dependencies are available and accessible from a flat source-/classpath. => Since a Java compiler is now also supposed to check access restrictions imposed by module declarations, the JLS also needs to specify this in depth, or it at least needs to point to JavaSE-9 APIs that contain the necessary specifications. See e.g. Stephan Hermann's questions about the meaning of a qualified name. Some examples: - "The ordinary compilation units that are visible to M are the observable ordinary compilation units associated with modules read by M. The host system must use the Java Platform Module System to determine which modules are read by M (?7.7.1)." => Neither "read" nor "Java Platform Module System" are specified anywhere. - "An implementation of the Java SE Platform must keep track of types within packages by the combination of their enclosing module names and their binary names (?13.1). Multiple ways of naming a type must be expanded to binary names to make sure that such names are understood as referring to the same type." =>What should happen if there are multiple types with the same binary name but different enclosing modules? Can they coexist or is this a compile error? JLS9 7.6 "Top Level Type Declarations" doesn't mention modules when it says: "It is a compile-time error if the name of a top level type appears as the name of any other top level class or interface type declared in the same package." Such things are relevant if you want to write a compliant compiler. E.g. JLS9 7.7 "Module Declarations" informally talks about "the modulepath" and "automatic modules", but neither of these concepts are explained any further. Automatic modules, unnamed modules, and their semantics must be specified in the JLS. The outdated http://openjdk.java.net/projects/jigsaw/spec/sotms/ has some more explanations, but since this is not part of the spec, it's irrelevant for a vote on JSR 376. => The JLS must either be self-contained or it must link to relevant other documents that are declared as equally dependable parts of the spec. The grammar for the module-info.java with its "restricted keywords" is highly problematic, since the language it defines is not processable by established compiler technology. Hacks are possible, but they are costly and prevent established error recovery techniques from working. These are only the concerns from a compiler and IDE implementer's perspective (i.e. answers whether we can implement something given the specs, not whether it's the right thing to do). We don't have a clear position on concerns voiced by implementers of existing Java modularity frameworks, but we're worried that the JPMS may not be suitable as a basis for those frameworks. Regards, The Eclipse JDT team From scolebourne at joda.org Thu May 4 21:21:38 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 4 May 2017 22:21:38 +0100 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170504103906.689386446@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: On 4 May 2017 at 18:38, wrote: > - Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose > value is used as the name of the automatic module defined by that JAR > file when it is placed on the module path, as previously suggested This is a step forward wrt automatic modules. > If a JAR file on the module path does not have such a > manifest attribute then its automatic-module name is computed using > the existing filename-based algorithm. Which we all agree is almost guaranteed to be wrong. > - To increase awareness of when automatic modules are used, and of the > consequences of their use, suggest that Java language compilers issue > two new types of warnings, and implement these warnings in the RI: Perhaps these should be reconsidered? If the module depends on an automatic module with a module name in the manifest, it can be considered reasonably stable. Whereas anything depending on a filename derived module name is unstable. That isn't being captured by the warnings. > In related threads on jigsaw-dev both Stephen Colebourne [e] and Robert > Scholte [f] suggest a new kind of explicit module, variously called > "partial" or "incomplete" or "soft", which can read all unnamed modules > so that dependencies that have not yet been modularized can be left on > the class path. This is an attractive notion on the surface, and in fact > one that we partly implemented in the prototype code base over two years > ago, using the term "loose". We didn't pursue it then, however, and I > don't think it's worth pursuing now, for two reasons: > > - If "partial" (or whatever) modules were to replace automatic modules > then application developers would have to wait until all of their > dependencies have been at least "partially" modularized before they > can even begin to experiment with modularizing their own code. This seems demonstrably false. module com.foo.application { requires com.foo.lib; } partial module com.foo.lib { exports com.foo.lib; // TODO: requires guava // TODO: requires joda-beans } modulepath = application classpath = guava, joda-beans The application developer can write the module declarations above, and use them even though all the external dependencies are yet to be modularised. It seems clear to me that the application developer has modularised their application, even if not completely. Given that this seems demonstrably false, I find the claim that "application developers would have to wait" rather odd. This suggests that something is perhaps not fully understood in the text above or the partial proposal. > - We could solve that by keeping automatic modules and adding partial > modules, but then we'd have a more complex module system and a more > complex story to tell. I agree with this. For me it either automatic or partial, not both. And partial is a lot better in technical and community terms IMO as it is a mechanism for reliable gradual migration. I still fail to see why dumping the jars of the classpath on the modulepath and hoping it works is a good strategy for migration - it seems like a bad short cut. Stephen On 4 May 2017 at 18:39, wrote: > Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and > Robert Scholte, for the extensive feedback. > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html > > TL;DR: Keep automatic modules, bring back the module-name JAR-file > manifest attribute, and strongly recommend reverse-DNS module names. > > Comments? > > - Mark From forax at univ-mlv.fr Thu May 4 22:16:05 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 04 May 2017 22:16:05 +0000 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <7B3DD855-466A-4BA9-9E39-41AF8606DCFB@univ-mlv.fr> On May 4, 2017 11:21:38 PM GMT+02:00, Stephen Colebourne wrote: >On 4 May 2017 at 18:38, wrote: ... >> - To increase awareness of when automatic modules are used, and of >the >> consequences of their use, suggest that Java language compilers >issue >> two new types of warnings, and implement these warnings in the >RI: > >Perhaps these should be reconsidered? If the module depends on an >automatic module with a module name in the manifest, it can be >considered reasonably stable. Whereas anything depending on a filename >derived module name is unstable. That isn't being captured by the >warnings. an automatic module is still not stable because it exports all packages by default while a real module may only export some of them. > >Stephen R?mi > > >On 4 May 2017 at 18:39, wrote: >> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and >> Robert Scholte, for the extensive feedback. >> >> >http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html >> >> TL;DR: Keep automatic modules, bring back the module-name JAR-file >> manifest attribute, and strongly recommend reverse-DNS module names. >> >> Comments? >> >> - Mark -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From mark.reinhold at oracle.com Fri May 5 00:19:58 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 04 May 2017 17:19:58 -0700 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <20170504171958.611894728@eggemoggin.niobe.net> 2017/5/4 14:21:38 -0700, Stephen Colebourne : > On 4 May 2017 at 18:38, mark.reinhold at oracle.com wrote: >> ... >> >> - Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose >> value is used as the name of the automatic module defined by that JAR >> file when it is placed on the module path, as previously suggested > > This is a step forward wrt automatic modules. > >> If a JAR file on the module path does not have such a >> manifest attribute then its automatic-module name is computed using >> the existing filename-based algorithm. > > Which we all agree is almost guaranteed to be wrong. ... but it's better than any of the alternatives. >> - To increase awareness of when automatic modules are used, and of the >> consequences of their use, suggest that Java language compilers issue >> two new types of warnings, and implement these warnings in the RI: > > Perhaps these should be reconsidered? If the module depends on an > automatic module with a module name in the manifest, it can be > considered reasonably stable. Whereas anything depending on a filename > derived module name is unstable. That isn't being captured by the > warnings. Named automatic modules still have unstable APIs, so I don't think the warnings need to be changed. Maybe they could distinguish "unstable name and API" vs. "unstable API only", but the opt-out warning for a `requires transitive` directive that refers to an automatic module, even if named, should be retained since that module's API is unstable. >> In related threads on jigsaw-dev both Stephen Colebourne [e] and Robert >> Scholte [f] suggest a new kind of explicit module, variously called >> "partial" or "incomplete" or "soft", which can read all unnamed modules >> so that dependencies that have not yet been modularized can be left on >> the class path. This is an attractive notion on the surface, and in fact >> one that we partly implemented in the prototype code base over two years >> ago, using the term "loose". We didn't pursue it then, however, and I >> don't think it's worth pursuing now, for two reasons: >> >> - If "partial" (or whatever) modules were to replace automatic modules >> then application developers would have to wait until all of their >> dependencies have been at least "partially" modularized before they >> can even begin to experiment with modularizing their own code. > > This seems demonstrably false. > > module com.foo.application { > requires com.foo.lib; > } > partial module com.foo.lib { > exports com.foo.lib; > // TODO: requires guava > // TODO: requires joda-beans > } > modulepath = application > classpath = guava, joda-beans > > The application developer can write the module declarations above, and > use them even though all the external dependencies are yet to be > modularised. It seems clear to me that the application developer has > modularised their application, even if not completely. By "modularizing" in the above point I meant "completely modularizing", but you're right: Application developers could start by writing their own partial modules. Is that really the best way, though, to encourage everyone to approach modularization? A useful aspect of the present design is that an explicit module is a complete module. If you've written a `module-info.java` file for your component then you're done. If explicit modules can be either partial or complete then I worry that many people will partially modularize a component but then never come back to completely modularize it. I also think it's essential for a developer new to all of this to be able to experiment immediately with both strong encapsulation (`exports`) and reliable configuration (`requires`). A story that says "write a partial module now, come back and make it complete later on after all of its dependencies have been (perhaps partially) modularized" is a story in which you won't be able to write and debug your `requires` directives until all of your dependencies have been (at least partially) modularized. That could be a pretty long story. - Mark From brianf at infinity.nu Fri May 5 00:58:31 2017 From: brianf at infinity.nu (Brian Fox) Date: Thu, 4 May 2017 20:58:31 -0400 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170504103906.689386446@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: On Thu, May 4, 2017 at 1:39 PM, wrote: > Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and > Robert Scholte, for the extensive feedback. > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/ > 2017-May/000687.html > > TL;DR: Keep automatic modules, bring back the module-name JAR-file > manifest attribute, and strongly recommend reverse-DNS module names. > Comments? > I think this is a tremendous development that will allow a path towards modularization and away from potential fragmentation of the Java ecosystem. Thank you for continuing to consider our feedback after what I'm sure has been a long slog to get this JSR over the finish line with a billion small sheds painted many colors. > > - Mark > From mark.reinhold at oracle.com Fri May 5 02:20:28 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 04 May 2017 19:20:28 -0700 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <20170504192028.881922934@eggemoggin.niobe.net> 2017/5/4 17:58:31 -0700, brianf at infinity.nu: > On Thu, May 4, 2017 at 1:39 PM, wrote: >> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and >> Robert Scholte, for the extensive feedback. >> >> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/ >> 2017-May/000687.html >> >> TL;DR: Keep automatic modules, bring back the module-name JAR-file >> manifest attribute, and strongly recommend reverse-DNS module names. >> >> Comments? > > I think this is a tremendous development that will allow a path towards > modularization and away from potential fragmentation of the Java ecosystem. > Thank you for continuing to consider our feedback after what I'm sure has > been a long slog to get this JSR over the finish line with a billion small > sheds painted many colors. Thank you for saying so, and thanks for your persistence! I've learned a lot. - Mark From rfscholte at apache.org Fri May 5 10:04:15 2017 From: rfscholte at apache.org (Robert Scholte) Date: Fri, 05 May 2017 12:04:15 +0200 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170504103906.689386446@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: Hi Mark, thanks for these adjustments. In general they look good, but it all depends on the details: Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose value is used as the name of the automatic module defined by that JAR file when it is placed on the module path, as previously suggested [2][3]. If a JAR file on the module path does not have such a manifest attribute then its automatic-module name is computed using the existing filename-based algorithm. Just to be clear: does this make it an auto module? In that case we're still a bit stuck, because I still insist that jars published to any public artifact repository should never refer to an automodule. Published jars with this attribute should have received this name by their developer and this also implies that the jar is Jigsaw-ready, e.g. no more split packages. In the future its users should be able to switch to the fully modular jar with the same name without any problem. So I need an extra state, e.g. isAutomatic() + isNamed(), in which case I have to change my insisting line: "Never publish jars to any public artifact repository that refers to *unnamed* automatic modules". A user of a library can suggest a stable name to the library's maintainer, and easily submit a patch that implements it. This will enable the uncoordinated modularization of libraries across the entire ecosystem. I don't think this is realistic. Jars live in the local repository and will stay there even during compilation. Is a user patches such jars, it'll effect all his local Java projects. Only when Maven will do both compiling and generation the distributable/deployable one could isolate jars. However, the concept is that plugins don't have any knowledge of each other and shouldn't share any information. In other words: maven-war-plugin is not aware of maven-compiler-plugin; the maven-war-plugin just collects all the content for the web archive, including the jars as specified in the pom.xml, pulling them from the local repository. In case the user is developing an application he might use this, but since he can already use automodules, I expect him to use that, it's less painful compared to patching its dependencies artifacts. In case the user is developing a library he cannot use this, nor can he refer to automodule, because he doesn't control the combinations of jars used by the application developer. thanks, Robert On Thu, 04 May 2017 19:39:06 +0200, wrote: > Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and > Robert Scholte, for the extensive feedback. > > http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html > > TL;DR: Keep automatic modules, bring back the module-name JAR-file > manifest attribute, and strongly recommend reverse-DNS module names. > > Comments? > > - Mark From scolebourne at joda.org Fri May 5 10:29:35 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 5 May 2017 11:29:35 +0100 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170504171958.611894728@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> <20170504171958.611894728@eggemoggin.niobe.net> Message-ID: On 5 May 2017 at 01:19, wrote: >> The application developer can write the module declarations above, and >> use them even though all the external dependencies are yet to be >> modularised. It seems clear to me that the application developer has >> modularised their application, even if not completely. > > By "modularizing" in the above point I meant "completely modularizing", Personally, I can't see how a module that depends on an automatic module can be described as "completely modularized" - it is built on sand, because the requires names are wrong (filenames). Having reflected on this overnight, I've come to the conclusion that the revised proposal still doesn't solve the basic community-level problem that is blocking adoption. Consider Joda-Convert that depends on Guava (optionally). With the revised proposal, I still can't release Joda-Convert until Guava has been made module aware (either by adding a full module declaration, or the MANIFEST attribute). The stack of dependencies is still forced to be upgraded from the bottom-up, which we agree won't work. If I write and release: module org.joda.convert { requires guava; } it will work today with guava as an automatic module. But when guava upgrades and becomes a complete module, it will stop working (because the module name will be different). Every project that depends on Joda-Convert that wants to upgrade Guava cannot do so until Joda-Convert does another release: module org.joda.convert { requires com.google.common; } As an approach, it causes every open source project to become tightly coupled, something that isn't feasible and which the community would have to reject. As such, I would not release Joda-Convert as a module until Guava is done. We'd be back to bottom-up migration. ie. all the manifest entry does is make the bottom-up migration a little quicker. Sadly, my view remains that the revised proposal is still broken when viewed through the open source community lens. My preference going forward remains partial modules, because they fit with the development workflow of many/most open source projects (and probably most closed source ones too). They get developed, with new features and bug fixes. And from time-to-time, dependencies get updated. When the dependency is updated, it would be a small extra step to add the missing "requires" line. I don't buy the need for "immediate experimentation" - quick things are usually short cuts that cause more harm than good. Since it appears unlikely partial modules will be considered, I'll send a separate thread with another attempt to mitigate the worst effects of automatic modules. Stephen From scolebourne at joda.org Fri May 5 10:36:38 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 5 May 2017 11:36:38 +0100 Subject: Explicit file names in module-info - #AutomaticModuleNames Message-ID: This is a proposal to mitigate some of the worst effects of automatic modules. Partial modules remain the better solution. We now have an agreed naming convention (reverse DNS), which is something we haven't had before. This gives us relatively stable and predictable names. But we also know that the module name will differ from the filename in most cases. As demonstrated before, having two disconnected names will block migration when forced to move from the filename to the module name. Thus, the biggest design problem is that the module depending on an automatic module is being asked to use the wrong name, with no link to the right one which they can now predict with reasonable certainty. As such, my proposal is that the module depending on an automatic module must specify the long-term stable module name, with the short-term filename included as additional metadata. Here is a keyword approach to this: module org.joda.convert { requires com.google.common file guava; } Here is an annotation approach that could be used instead: @Alias(moduleName = "com.google.common" fileName="guava") module org.joda.convert { requires com.google.common; } With both of the above, the module declaration uses the correct final module name from day one, which is vital to not creating future problems. The long-term stable name can be communicated in many ways, such as on a website or by email, not just by code or manifest. I can now release Joda-Convert, whether or not Guava has been modularised. Success!!!! The short-term migration-only nature of the filename is much much clearer. The filename can be removed once the dependency is a module (and static analysis tools could highlight that). The fileName would be the Maven artifactId in most cases. (There is a case for the fileName to be a list of possible filenames for additional flexibility). I think this design addresses some of Roberts's concern too. With this plan, Maven Central would contain modules depending on automatic modules, but the dependency names would be sufficiently stable for this not to be the major issue it has been previously. While I don't think automatic modules are the best option, were the above chosen, I think it would be a solution the community could successfully and easily adopt. Stephen From forax at univ-mlv.fr Fri May 5 13:16:44 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 5 May 2017 15:16:44 +0200 (CEST) Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <567927167.2960851.1493990204131.JavaMail.zimbra@u-pem.fr> Hi Robert, ----- Mail original ----- > De: "Robert Scholte" > ?: "mark reinhold" > Cc: jigsaw-dev at openjdk.java.net > Envoy?: Vendredi 5 Mai 2017 12:04:15 > Objet: Re: Revised proposal for #AutomaticModuleNames > Hi Mark, > > thanks for these adjustments. In general they look good, but it all > depends on the details: > > Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose > value is used as the name of the automatic module defined by that JAR > file when it is placed on the module path, as previously suggested > [2][3]. If a JAR file on the module path does not have such a > manifest attribute then its automatic-module name is computed using > the existing filename-based algorithm. > > Just to be clear: does this make it an auto module? In that case we're > still a bit stuck, because I still insist that jars published to any > public artifact repository should never refer to an automodule. Published > jars with this attribute should have received this name by their developer > and this also implies that the jar is Jigsaw-ready, e.g. no more split > packages. In the future its users should be able to switch to the fully > modular jar with the same name without any problem. So I need an extra > state, e.g. isAutomatic() + isNamed(), in which case I have to change my > insisting line: "Never publish jars to any public artifact repository that > refers to *unnamed* automatic modules". yes, knowing if the name is auto-calculated or not may be interesting, that's said, you can already get the manifest file from the ModuleReader and check if there is an entry named Automatic-Module-Name so i'm not sure surfacing isNamed() in the API is a good idea. > > A user of a library can suggest a stable name to the library's > maintainer, and easily submit a patch that implements it. This will > enable the uncoordinated modularization of libraries across the > entire ecosystem. > > I don't think this is realistic. Jars live in the local repository and > will stay there even during compilation. Is a user patches such jars, > it'll effect all his local Java projects. Only when Maven will do both > compiling and generation the distributable/deployable one could isolate > jars. [...] I think there is a misunderstanding here, i read 'easily submit a patch' as 'easily submit a pull request' that will change the source so the next time the jar is published on central, it will have a module name in its manifest. > > thanks, > Robert R?mi > > On Thu, 04 May 2017 19:39:06 +0200, wrote: > >> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and >> Robert Scholte, for the extensive feedback. >> >> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html >> >> TL;DR: Keep automatic modules, bring back the module-name JAR-file >> manifest attribute, and strongly recommend reverse-DNS module names. >> >> Comments? >> > > - Mark From robert.j.saulnier at gmail.com Fri May 5 14:19:38 2017 From: robert.j.saulnier at gmail.com (Robert J. Saulnier) Date: Fri, 5 May 2017 11:19:38 -0300 Subject: Can automatic modules be made to work? Message-ID: I've only somewhat followed this discussion. The following might be nonsensical or already discussed. The issue with automatic modules is we don't know what the names of future modules will be, but what we do know is what packages we require. So instead of guessing a module name, we could require package(s) until the actual module becomes available. module my.module { exports ...; requires ...; requires package org.junit; requires package org.junit.rules; } So in the above example, it would need to check if a module on the module-path exports the packages listed, if not, look in the Jar files for the packages. Once Junit modularizes their stuff, we can update our module at our leisure: module my.module { exports ...; requires ...; requires ; } Bob From mark.reinhold at oracle.com Fri May 5 14:41:55 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Fri, 05 May 2017 07:41:55 -0700 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <20170505074155.424443679@eggemoggin.niobe.net> 2017/5/5 3:04:15 -0700, Robert Scholte : > thanks for these adjustments. In general they look good, but it all > depends on the details: > > Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose > value is used as the name of the automatic module defined by that JAR > file when it is placed on the module path, as previously suggested > [2][3]. If a JAR file on the module path does not have such a > manifest attribute then its automatic-module name is computed using > the existing filename-based algorithm. > > Just to be clear: does this make it an auto module? Yes, except that its name is taken as the value of the attribute rather than computed from the JAR file's name. > In that case we're > still a bit stuck, because I still insist that jars published to any > public artifact repository should never refer to an automodule. Published > jars with this attribute should have received this name by their developer > and this also implies that the jar is Jigsaw-ready, e.g. no more split > packages. If a JAR file works on the module path as an automatic module then it already can't have any split packages. It doesn't matter whether its name is given in the attribute or computed from the filename. > In the future its users should be able to switch to the fully > modular jar with the same name without any problem. So I need an extra > state, e.g. isAutomatic() + isNamed(), in which case I have to change my > insisting line: "Never publish jars to any public artifact repository that > refers to *unnamed* automatic modules". All automatic modules are named, either from the above attribute or from the filename. That is, `isAutomatic() && isNamed()` will be `true` for every automatic module. (Note that `isAutomatic()` is a method on `ModuleDescriptor` objects, while `isNamed()` is a method on run-time `Module` objects. The only "unnamed" modules are those that are associated with each class loader, for compatibility purposes.) I suspect what you really mean is "Never publish JARs that refer to automatic modules that do not have `Automatic-Module-Name` attributes". In general that's good advice. It might even be reasonable for managers of artifact repositories to insist upon it, although I now understand that that could be problematic for a repository as popular as Maven Central. > A user of a library can suggest a stable name to the library's > maintainer, and easily submit a patch that implements it. This will > enable the uncoordinated modularization of libraries across the > entire ecosystem. > > I don't think this is realistic. Jars live in the local repository and > will stay there even during compilation. Is a user patches such jars, > it'll effect all his local Java projects. As R?mi points out nearby, by "submit a patch" I meant "submit a pull request" or "send a source-code patch to the maintainer". If the maintainer likes the patch then they can merge it and it will show up in the next release. If they don't like it then at least they'll know that some of their users care, and perhaps be motivated to choose the (reverse DNS!) module name that they prefer and merge that. - Mark From rfscholte at apache.org Fri May 5 19:01:58 2017 From: rfscholte at apache.org (Robert Scholte) Date: Fri, 05 May 2017 21:01:58 +0200 Subject: Can automatic modules be made to work? In-Reply-To: References: Message-ID: Yes, this option has been mentioned in some form. The good thing about this part is that exports and requires use the same structure, i.e. the package is the unit shared between 2 modules. However, this will probably lead to a long list of requirements, so I understand the choice the name of the bundle of exported packages, being the module name. Also keep in mind that one of the fundamental choices made is that the module descriptor should not have mechanisms for migration path. That said, one might think that the partial requirements or soft/loose modules are there also for migration. I would disagree with that. I expect that some jars will never become a module or will ever be re-released just for specifying a Automatic-Module-Name attribute, in which case you must refer to them as auto modules. In such cases you cannot talk about migration but about the fact that projects will always depend on unmodularized jars. One solution was the soft/loose modules proposal which I think is valid but it all depends on the opinion if this "pattern" will be there just for migration or not. Robert On Fri, 05 May 2017 16:19:38 +0200, Robert J. Saulnier wrote: > I've only somewhat followed this discussion. The following might be > nonsensical or already discussed. > > The issue with automatic modules is we don't know what the names of > future > modules will be, but what we do know is what packages we require. So > instead of guessing a module name, we could require package(s) until the > actual module becomes available. > > module my.module { > > exports ...; > requires ...; > > requires package org.junit; > requires package org.junit.rules; > } > > So in the above example, it would need to check if a module on the > module-path exports the packages listed, if not, look in the Jar files > for > the packages. > > Once Junit modularizes their stuff, we can update our module at our > leisure: > > module my.module { > > exports ...; > requires ...; > > requires ; > } > > Bob From rfscholte at apache.org Fri May 5 19:18:22 2017 From: rfscholte at apache.org (Robert Scholte) Date: Fri, 05 May 2017 21:18:22 +0200 Subject: Explicit file names in module-info - #AutomaticModuleNames In-Reply-To: References: Message-ID: On Fri, 05 May 2017 12:36:38 +0200, Stephen Colebourne wrote: > I think this design addresses some of Roberts's concern too. With this > plan, Maven Central would contain modules depending on automatic > modules, but the dependency names would be sufficiently stable for > this not to be the major issue it has been previously. > While I don't think automatic modules are the best option, were the > above chosen, I think it would be a solution the community could > successfully and easily adopt. My most favorite solution is still the support for loose/soft modules. The pros: it'll only introduce one new keyword; the cons: the code might not compile/run *upfront* due to missing jars, however I doubt if this is really an issue assuming most Java projects use tooling to solve this. Adding mapper information to the module-descriptor is an option, but that'll add the automodule name in some way to the descriptor, and I'd like to avoid that because it should not be used as inspiration to pick your module name. From alan.bateman at oracle.com Fri May 5 20:07:56 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 05 May 2017 20:07:56 +0000 Subject: hg: jigsaw/jake/jdk: Automatic-Module-Name Message-ID: <201705052007.v45K7uTv018605@aojmv0008.oracle.com> Changeset: d6c11ff1e0a9 Author: alanb Date: 2017-05-05 21:05 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/d6c11ff1e0a9 Automatic-Module-Name ! src/java.base/share/classes/java/lang/module/ModuleFinder.java ! src/java.base/share/classes/jdk/internal/module/ModulePath.java ! test/java/lang/module/AutomaticModulesTest.java From alan.bateman at oracle.com Fri May 5 20:08:03 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 05 May 2017 20:08:03 +0000 Subject: hg: jigsaw/jake/langtools: Automatic-Module-Name Message-ID: <201705052008.v45K83NN018666@aojmv0008.oracle.com> Changeset: 822101a1a31d Author: alanb Date: 2017-05-05 21:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/822101a1a31d Automatic-Module-Name ! src/jdk.compiler/share/classes/com/sun/tools/javac/file/Locations.java From alex.buckley at oracle.com Fri May 5 20:22:24 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 05 May 2017 13:22:24 -0700 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: References: Message-ID: <590CDF00.9010306@oracle.com> On 5/4/2017 11:51 AM, Markus Keller wrote: > The other big thing is that the JLS draft specifies the syntax of the > module-info.java file, but it is quite vague about the semantics of > modules. > > JLS9 12 "Execution" explains how class loading is supposed to work in the > JVM, but it's unclear how modules and and their access restrictions should > come into the picture here. Layers are not even mentioned anywhere. Before > Java 9, classloading and discovery of .class files was only a run-time > concern. During compilation, the assumption was that all dependencies are > available and accessible from a flat source-/classpath. > => Since a Java compiler is now also supposed to check access restrictions > imposed by module declarations, the JLS also needs to specify this in > depth, or it at least needs to point to JavaSE-9 APIs that contain the > necessary specifications. See e.g. Stephan Hermann's questions about the > meaning of a qualified name. Chapter 12 of the JLS contains a variety of material: - 12.1, 12.2, and 12.3 cover run time behavior, not compile time behavior, so I'm not sure why the Eclipse compiler relies on these sections. For example, layers are not part of the Java language so there is nothing to mandate about them for a compiler. Looking forward, there are enhancements to the corresponding JVMS9 sections (5.2 and 5.3) which may yet result in enhancements to these JLS sections, but that is "editorial"; it will not affect the behavior of a compiler. - 12.4, 12.5, and 12.6 are normative for a compiler but are unrelated to the module system. - 12.7 and 12.8 are normative but for a JVM implementation not a compiler. > Some examples: > - "The ordinary compilation units that are visible to M are the > observable ordinary compilation units associated with modules read by M. > The host > system must use the Java Platform Module System to determine which modules > are read by M (?7.7.1)." > => Neither "read" nor "Java Platform Module System" are specified > anywhere. This was raised in the jigsaw-dev thread "Java Platform Module System" [1] and discussed at length there. The Java Platform Module System is as much part of the Java SE Platform as the Java language, so the Java Language Specification is able to depend upon the specification of the Java Platform Module System (which happens to be presented in API form rather than as a narrative document). [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012301.html > - "An implementation of the Java SE Platform must keep track of types > within > packages by the combination of their enclosing module names and their > binary > names (?13.1). Multiple ways of naming a type must be expanded to binary > names > to make sure that such names are understood as referring to the same > type." > =>What should happen if there are multiple types with the same binary name > but different enclosing modules? Can they coexist or is this a compile > error? JLS9 7.6 "Top Level Type Declarations" doesn't mention modules when > it says: "It is a compile-time error if the name of a top level type > appears as the name of any other top level class or interface type > declared in the same package." This was discussed in the same thread, and I made concrete suggestions for how to clarify the matter [2]. [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012488.html > Such things are relevant if you want to write a compliant compiler. > > E.g. JLS9 7.7 "Module Declarations" informally talks about "the > modulepath" and "automatic modules", but neither of these concepts are > explained any further. Automatic modules, unnamed modules, and their > semantics must be specified in the JLS. The outdated > http://openjdk.java.net/projects/jigsaw/spec/sotms/ has some more > explanations, but since this is not part of the spec, it's irrelevant for > a vote on JSR 376. > > => The JLS must either be self-contained or it must link to relevant other > documents that are declared as equally dependable parts of the spec. This was discussed in the same thread, and appeared to be clarified [3]. [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012425.html > The grammar for the module-info.java with its "restricted keywords" is > highly problematic, since the language it defines is not processable by > established compiler technology. Hacks are possible, but they are costly > and prevent established error recovery techniques from working. This was discussed in the same thread, and appeared to be non-blocking for JDT [4]. The JLS has not guaranteed for many years that the grammar of the Java language is aligned with the capabilities of pre-existing tools. I recall that parsing lambda expressions presented a similar kind of challenge to JDT, and that admirable solutions were found [5]. [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012500.html [5] https://www.eclipsecon.org/na2014/session/jdt-embraces-lambda-expressions.html Alex From brianf at infinity.nu Fri May 5 21:14:49 2017 From: brianf at infinity.nu (Brian Fox) Date: Fri, 5 May 2017 17:14:49 -0400 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170505074155.424443679@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> <20170505074155.424443679@eggemoggin.niobe.net> Message-ID: On Fri, May 5, 2017 at 10:41 AM, wrote: > I suspect what you really mean is "Never publish JARs that refer to > automatic modules that do not have `Automatic-Module-Name` attributes". > In general that's good advice. > +1 For me, the point of the attribute was to allow an intermediate step. > It might even be reasonable for managers > of artifact repositories to insist upon it, although I now understand > that that could be problematic for a repository as popular as Maven > Central. > We can look at encouraging it from the Maven side at least. Hard stops on the repo can be a challenge as described previously. From stephan.herrmann at berlin.de Sat May 6 16:56:12 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Sat, 6 May 2017 18:56:12 +0200 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <590CDF00.9010306@oracle.com> References: <590CDF00.9010306@oracle.com> Message-ID: Alex, I appreciate your answers to our questions, which give hope that a future version - incorporating all this - will be sufficient for defining what is Java 9 from a compiler's perspective. The post sent by Markus explicitly refers to the specification as it was submitted for public review, which is not sufficient in several regards. I see two reasons for insisting in this distinction: It is necessary to call out that JSR 376 is again behind schedule, putting third party implementors under extreme time pressure, to the degree that a compliant implementation may not be possible on the currently scheduled release date. Only once we hold in our hands a specification that has all the missing parts integrated and that has passed some level of QA, it is possible to confirm whether all the pieces fit together in a coherent, consistent and sufficient way. (This is my personal view as an individual contributor to Eclipse JDT) Stephan On 05.05.2017 22:22, Alex Buckley wrote: > On 5/4/2017 11:51 AM, Markus Keller wrote: >> The other big thing is that the JLS draft specifies the syntax of the >> module-info.java file, but it is quite vague about the semantics of >> modules. >> >> JLS9 12 "Execution" explains how class loading is supposed to work in the >> JVM, but it's unclear how modules and and their access restrictions should >> come into the picture here. Layers are not even mentioned anywhere. Before >> Java 9, classloading and discovery of .class files was only a run-time >> concern. During compilation, the assumption was that all dependencies are >> available and accessible from a flat source-/classpath. >> => Since a Java compiler is now also supposed to check access restrictions >> imposed by module declarations, the JLS also needs to specify this in >> depth, or it at least needs to point to JavaSE-9 APIs that contain the >> necessary specifications. See e.g. Stephan Hermann's questions about the >> meaning of a qualified name. > > Chapter 12 of the JLS contains a variety of material: > > - 12.1, 12.2, and 12.3 cover run time behavior, not compile time behavior, so I'm not sure why the Eclipse compiler relies on these > sections. For example, layers are not part of the Java language so there is nothing to mandate about them for a compiler. Looking > forward, there are enhancements to the corresponding JVMS9 sections (5.2 and 5.3) which may yet result in enhancements to these JLS > sections, but that is "editorial"; it will not affect the behavior of a compiler. > > - 12.4, 12.5, and 12.6 are normative for a compiler but are unrelated to the module system. > > - 12.7 and 12.8 are normative but for a JVM implementation not a compiler. > >> Some examples: >> - "The ordinary compilation units that are visible to M are the >> observable ordinary compilation units associated with modules read by M. >> The host >> system must use the Java Platform Module System to determine which modules >> are read by M (?7.7.1)." >> => Neither "read" nor "Java Platform Module System" are specified >> anywhere. > > This was raised in the jigsaw-dev thread "Java Platform Module System" [1] and discussed at length there. The Java Platform Module > System is as much part of the Java SE Platform as the Java language, so the Java Language Specification is able to depend upon the > specification of the Java Platform Module System (which happens to be presented in API form rather than as a narrative document). > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012301.html > >> - "An implementation of the Java SE Platform must keep track of types >> within >> packages by the combination of their enclosing module names and their >> binary >> names (?13.1). Multiple ways of naming a type must be expanded to binary >> names >> to make sure that such names are understood as referring to the same >> type." >> =>What should happen if there are multiple types with the same binary name >> but different enclosing modules? Can they coexist or is this a compile >> error? JLS9 7.6 "Top Level Type Declarations" doesn't mention modules when >> it says: "It is a compile-time error if the name of a top level type >> appears as the name of any other top level class or interface type >> declared in the same package." > > This was discussed in the same thread, and I made concrete suggestions for how to clarify the matter [2]. > > [2] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012488.html > >> Such things are relevant if you want to write a compliant compiler. >> >> E.g. JLS9 7.7 "Module Declarations" informally talks about "the >> modulepath" and "automatic modules", but neither of these concepts are >> explained any further. Automatic modules, unnamed modules, and their >> semantics must be specified in the JLS. The outdated >> http://openjdk.java.net/projects/jigsaw/spec/sotms/ has some more >> explanations, but since this is not part of the spec, it's irrelevant for >> a vote on JSR 376. >> >> => The JLS must either be self-contained or it must link to relevant other >> documents that are declared as equally dependable parts of the spec. > > This was discussed in the same thread, and appeared to be clarified [3]. > > [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012425.html > >> The grammar for the module-info.java with its "restricted keywords" is >> highly problematic, since the language it defines is not processable by >> established compiler technology. Hacks are possible, but they are costly >> and prevent established error recovery techniques from working. > > This was discussed in the same thread, and appeared to be non-blocking for JDT [4]. The JLS has not guaranteed for many years that > the grammar of the Java language is aligned with the capabilities of pre-existing tools. I recall that parsing lambda expressions > presented a similar kind of challenge to JDT, and that admirable solutions were found [5]. > > [4] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012500.html > [5] https://www.eclipsecon.org/na2014/session/jdt-embraces-lambda-expressions.html > > Alex From stephan.herrmann at berlin.de Sat May 6 17:13:51 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Sat, 6 May 2017 19:13:51 +0200 Subject: Reminder to update JVMS Message-ID: I just happened to search for the specification of the class file representation of modules. I was quite surprised to find nothing in JVMS. Then I found: "The changes to the class-file chapter in support of module declarations are not included in this draft of the JVMS; they will be included in a future version of this specification." Makes me wonder, if this is still work in progress? Anyway, please don't forget :) Stephan From mark.reinhold at oracle.com Sat May 6 19:49:44 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Sat, 06 May 2017 12:49:44 -0700 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: References: <590CDF00.9010306@oracle.com> Message-ID: <20170506124944.217983127@eggemoggin.niobe.net> 2017/5/6 9:56:12 -0700, stephan.herrmann at berlin.de: > Alex, > > I appreciate your answers to our questions, which give hope > that a future version - incorporating all this - will be sufficient > for defining what is Java 9 from a compiler's perspective. > > The post sent by Markus explicitly refers to the specification > as it was submitted for public review, which is not sufficient > in several regards. > > I see two reasons for insisting in this distinction: > > It is necessary to call out that JSR 376 is again behind schedule, > putting third party implementors under extreme time pressure, > to the degree that a compliant implementation may not be possible > on the currently scheduled release date. > > Only once we hold in our hands a specification that has all the > missing parts integrated and that has passed some level of QA, > it is possible to confirm whether all the pieces fit together > in a coherent, consistent and sufficient way. > > (This is my personal view as an individual contributor to Eclipse JDT) Stephan, Thanks for expressing your concerns. The maintainers of many libraries, frameworks, and tools have been able to keep up with the latest developments in JPMS and Jigsaw in real time, on this list and elsewhere. Some of them are ready for JDK 9 now, and more will be ready by JDK 9 GA or shortly thereafter. You and Markus seem to be saying that since the Eclipse JDT team can only really get started when they have a specification that's complete in every detail then the rest of the Java community must wait. Is that what you mean? I understand that Eclipse is important, but is it important enough to hold up the release for everyone else? - Mark From stephan.herrmann at berlin.de Sat May 6 20:51:32 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Sat, 6 May 2017 22:51:32 +0200 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <20170506124944.217983127@eggemoggin.niobe.net> References: <590CDF00.9010306@oracle.com> <20170506124944.217983127@eggemoggin.niobe.net> Message-ID: <75b9bd01-2d7b-8fa3-389d-e28629b2a8ae@berlin.de> On 06.05.2017 21:49, mark.reinhold at oracle.com wrote: > 2017/5/6 9:56:12 -0700, stephan.herrmann at berlin.de: >> Alex, >> >> I appreciate your answers to our questions, which give hope >> that a future version - incorporating all this - will be sufficient >> for defining what is Java 9 from a compiler's perspective. >> >> The post sent by Markus explicitly refers to the specification >> as it was submitted for public review, which is not sufficient >> in several regards. >> >> I see two reasons for insisting in this distinction: >> >> It is necessary to call out that JSR 376 is again behind schedule, >> putting third party implementors under extreme time pressure, >> to the degree that a compliant implementation may not be possible >> on the currently scheduled release date. >> >> Only once we hold in our hands a specification that has all the >> missing parts integrated and that has passed some level of QA, >> it is possible to confirm whether all the pieces fit together >> in a coherent, consistent and sufficient way. >> >> (This is my personal view as an individual contributor to Eclipse JDT) > > Stephan, > > Thanks for expressing your concerns. > > The maintainers of many libraries, frameworks, and tools have been able > to keep up with the latest developments in JPMS and Jigsaw in real time, > on this list and elsewhere. Some of them are ready for JDK 9 now, and > more will be ready by JDK 9 GA or shortly thereafter. > > You and Markus seem to be saying that since the Eclipse JDT team can > only really get started when they have a specification that's complete > in every detail then the rest of the Java community must wait. Is that > what you mean? > > I understand that Eclipse is important, but is it important enough to > hold up the release for everyone else? Mark, I am honestly glad that my role in all this is not at a management level, but at the technical level, so I humbly refrain from answering your questions. Still, I feel qualified and obliged to add my share of transparency to the discussion. If a spec is insufficient, I will call it insufficient. Alex knows that I'm doing so in deep appreciation of JLS. In many discussions about differences between javac and ecj I defended the view that "right" or "wrong" is not defined by any implementation, but only by JLS. I keep making a point that implementing ecj based on the specification and nothing but the specification is the ultimate quality assurance that could be applied to JLS. Please help me continuing this advocacy of JLS. best, Stephan From mike.milinkovich at eclipse.org Sat May 6 21:27:27 2017 From: mike.milinkovich at eclipse.org (Mike Milinkovich) Date: Sat, 6 May 2017 17:27:27 -0400 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <20170506124944.217983127@eggemoggin.niobe.net> References: <590CDF00.9010306@oracle.com> <20170506124944.217983127@eggemoggin.niobe.net> Message-ID: <1DBAB83B-D775-4FA9-95E7-2DA0532143E0@eclipse.org> Mark, You are comparing apples to oranges. The Eclipse Compiler for Java is neither a tool, nor a framework. It is the other Java compiler. The one that needs to actually rely on the specification, as opposed to Oracle's implementation. The whole purpose of the JCP is to enable independent implementations, such as ecj. If the specs don't do so, we might as well just disband the whole facade. The Eclipse JDT team has worked extremely hard throughout the entire Java 9 development cycle to overcome incomplete and ambiguous specifications. Their efforts have made the spec much better than it would be otherwise. I can appreciate the schedule and business pressures you're under, but your attempts to shift the blame for this situation to others are counter-productive. Mike Milinkovich mike.milinkovich at eclipse.org +1.613.220.3223 > On May 6, 2017, at 3:49 PM, mark.reinhold at oracle.com wrote: > > 2017/5/6 9:56:12 -0700, stephan.herrmann at berlin.de: >> Alex, >> >> I appreciate your answers to our questions, which give hope >> that a future version - incorporating all this - will be sufficient >> for defining what is Java 9 from a compiler's perspective. >> >> The post sent by Markus explicitly refers to the specification >> as it was submitted for public review, which is not sufficient >> in several regards. >> >> I see two reasons for insisting in this distinction: >> >> It is necessary to call out that JSR 376 is again behind schedule, >> putting third party implementors under extreme time pressure, >> to the degree that a compliant implementation may not be possible >> on the currently scheduled release date. >> >> Only once we hold in our hands a specification that has all the >> missing parts integrated and that has passed some level of QA, >> it is possible to confirm whether all the pieces fit together >> in a coherent, consistent and sufficient way. >> >> (This is my personal view as an individual contributor to Eclipse JDT) > > Stephan, > > Thanks for expressing your concerns. > > The maintainers of many libraries, frameworks, and tools have been able > to keep up with the latest developments in JPMS and Jigsaw in real time, > on this list and elsewhere. Some of them are ready for JDK 9 now, and > more will be ready by JDK 9 GA or shortly thereafter. > > You and Markus seem to be saying that since the Eclipse JDT team can > only really get started when they have a specification that's complete > in every detail then the rest of the Java community must wait. Is that > what you mean? > > I understand that Eclipse is important, but is it important enough to > hold up the release for everyone else? > > - Mark From mark.reinhold at oracle.com Sat May 6 21:38:26 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Sat, 06 May 2017 14:38:26 -0700 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <75b9bd01-2d7b-8fa3-389d-e28629b2a8ae@berlin.de> References: <590CDF00.9010306@oracle.com> <20170506124944.217983127@eggemoggin.niobe.net> <75b9bd01-2d7b-8fa3-389d-e28629b2a8ae@berlin.de> Message-ID: <20170506143826.914298919@eggemoggin.niobe.net> 2017/5/6 13:51:32 -0700, stephan.herrmann at berlin.de: > Mark, > > I am honestly glad that my role in all this is not at a management level, > but at the technical level, so I humbly refrain from answering your questions. Understood. > Still, I feel qualified and obliged to add my share of transparency to > the discussion. If a spec is insufficient, I will call it insufficient. I would expect nothing less! > Alex knows that I'm doing so in deep appreciation of JLS. In many > discussions about differences between javac and ecj I defended the view > that "right" or "wrong" is not defined by any implementation, but only > by JLS. I completely agree. > I keep making a point that implementing ecj based on the > specification and nothing but the specification is the ultimate quality > assurance that could be applied to JLS. Your team's work on ecj over the years has definitely helped to improve the JLS. I really appreciate that, and I hope that the rest of the community does too. - Mark From mark.reinhold at oracle.com Sat May 6 22:30:41 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Sat, 06 May 2017 15:30:41 -0700 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <1DBAB83B-D775-4FA9-95E7-2DA0532143E0@eclipse.org> References: <590CDF00.9010306@oracle.com> <20170506124944.217983127@eggemoggin.niobe.net> <1DBAB83B-D775-4FA9-95E7-2DA0532143E0@eclipse.org> Message-ID: <20170506153041.482217656@eggemoggin.niobe.net> 2017/5/6 14:27:27 -0700, mike.milinkovich at eclipse.org: > Mark, > > You are comparing apples to oranges. The Eclipse Compiler for Java is > neither a tool, nor a framework. A compiler is a particular kind of software-development tool, last I checked. An important one, certainly, but a tool nonetheless. > It is the other Java compiler. The > one that needs to actually rely on the specification, as opposed to > Oracle's implementation. As I said in my nearby reply to Stephen, I very much appreciate that and I hope that others do as well. It's not clear to me, however, that its role in that regard justifies a delay to the release. > The whole purpose of the JCP is to enable independent implementations, > such as ecj. If the specs don't do so, we might as well just disband > the whole facade. > > The Eclipse JDT team has worked extremely hard throughout the entire > Java 9 development cycle to overcome incomplete and ambiguous > specifications. Their efforts have made the spec much better than it > would be otherwise. I have no doubt of that, and I appreciate their efforts. > I can appreciate the schedule and business pressures you're under, I'm not under any business pressure to ship Java SE 9 by the current scheduled date. If there's a very strong reason to slip the release then I will propose to slip the release, as I've always said -- and done. > but > your attempts to shift the blame for this situation to others are > counter-productive. I'm not trying to shift any blame anywhere. I'm merely trying to understand the view of the Eclipse JDT team, or the Eclipse Foundation more generally, for which you speak. So: Is it the view of the Eclipse Foundation that any release of the Java SE Platform that changes the Java programming language must wait until the maintainers of the Eclipse Compiler for Java are satisfied with the draft Java Language Specification? If not, then how would you characterize the conditions under which an update to the JLS may be considered ready for release? - Mark From peter.levart at gmail.com Sun May 7 08:37:34 2017 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 May 2017 10:37:34 +0200 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <5907D11D.20609@oracle.com> References: <5907D11D.20609@oracle.com> Message-ID: <1f806cdc-a241-04aa-6e4b-0b11abb1f0a9@gmail.com> Hi Kevin, On 05/02/2017 02:21 AM, Kevin Rushforth wrote: > This review is being cross-posted to both openjfx-dev and jigsaw-dev. > > Please review the proposed fix for: > > https://bugs.openjdk.java.net/browse/JDK-8177566 > http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ > > Details of the fix as well as notes to reviewers are in the bug report > [1] (e.g., I've also generated separate webrevs for the fix itself, > the doc changes, and the test changes). > > -- Kevin > > [1] > https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 > I think it is very unusual to handle unqualified exports as something special, different from qualified exports. I know what the reasoning is: if a package is exported unconditionally (to everyone) then it is part of public API and so the trampoline may access members of that package on behalf of anyone. But such reasoning is just a consequence of the lack of a finer-grained (per-module) access support in JavaFX. I know it is too much to ask for JDK 9, but could JavaFX in say JDK 10, given current API, somehow determine on whose behalf it is making the trampoline access? If it could, then the trampoline could allow qualified exports to be effective too. Regards, Peter From forax at univ-mlv.fr Sun May 7 23:06:46 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 8 May 2017 01:06:46 +0200 (CEST) Subject: --ignore-signing-information do not suppress all errors Message-ID: <1088807201.37376.1494198406792.JavaMail.zimbra@u-pem.fr> Hi all, I wonder if it's an issue or if there is a rational for not suppressing this error when using --ignore-signing-information. I'm patching the compiler related modules of the jdk9 with the one from amber/langtools, so i know that i'm doing something borderline but i gently ask jlink to do not bother me with the hash signature. ... jlink --module-path ./target/main/artifact:./deps:/usr/jdk/jdk-9-b167/jmods --add-modules jdk.policytool,jdk.charsets,jdk.pack,jdk.xml.dom,java.xml,jdk.deploy,java.datatransfer,java.jnlp,java.desktop,com.github.forax.pro.uberbooter,java.se,jdk.snmp,java.security.sasl,jdk.deploy.controlpanel,jdk.zipfs,java.base,jdk.crypto.ec,com.github.forax.pro.main,jdk.management.agent,jdk.jshell,jdk.editpad,oracle.desktop,jdk.jsobject,jdk.unsupported,java.smartcardio,jdk.scripting.nashorn,java.security.jgss,com.github.forax.pro.plugin.convention,jdk.dynalink,java.activation,java.sql,java.logging,jdk.jfr,jdk.internal.vm.ci,com.github.forax.pro.plugin.packager,jdk.packager.services,jdk.net,javafx.controls,jdk.internal.ed,jdk.compiler,jdk.naming.rmi,jdk.jconsole,jdk.internal.le,jdk.packager,jdk.jdwp.agent,jdk.internal.vm.compiler,com.github.forax.pro.daemon.imp,java.scripting,jdk.jartool,java.rmi,jdk.jdi,jdk.javaws,jdk.rmic,com.github.forax.pro.plugin.resolver,jdk.jstatd,jdk.httpserver,java.se.ee,jdk.jcmd,javafx.base,com.github.forax.pro.plugin.linker,com.github.forax.pro.plugin.compiler,jdk.plugin.dom,com.github.forax.pro.plugin.modulefixer,jdk.javadoc,jdk.xml.ws,java.sql.rowset,jdk.sctp,javafx.swing,jdk.jlink,jdk.scripting.nashorn.shell,oracle.net,java.xml.bind,java.compiler,javafx.graphics,javafx.fxml,jdk.plugin,javafx.media,jdk.accessibility,jdk.security.jgss,javafx.web,com.github.forax.pro.plugin.uberpackager,jdk.hotspot.agent,javafx.deploy,java.xml.crypto,jdk.incubator.httpclient,jdk.plugin.server,jdk.crypto.cryptoki,java.naming,java.prefs,jdk.internal.opt,jdk.attach,java.transaction,java.xml.ws,java.xml.ws.annotation,java.management,jdk.xml.bind,jdk.internal.jvmstat,java.instrument,jdk.management,jdk.security.auth,jdk.jdeps,jdk.aot,java.corba,java.management.rmi,jdk.naming.dns,jdk.localedata --launcher pro=com.github.forax.pro.main/com.github.forax.pro.main.Main --compress 0 --ignore-signing-information --output ./target/image Error: Hash of jdk.javadoc (5b3a628bce2aad807392ea8a3f2ce69a652e885e33db586694b47fcf6f5ebf52) differs to expected hash (6bdf7f26f72c9f102c5052f8ddf68a80a4b18f997a16e56805e3acc7347469b5) recorded in java.base cheers, R?mi From Alan.Bateman at oracle.com Mon May 8 06:24:51 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 May 2017 07:24:51 +0100 Subject: --ignore-signing-information do not suppress all errors In-Reply-To: <1088807201.37376.1494198406792.JavaMail.zimbra@u-pem.fr> References: <1088807201.37376.1494198406792.JavaMail.zimbra@u-pem.fr> Message-ID: <9bd0491f-4e0b-10d5-7a6b-ab958c8b95e0@oracle.com> On 08/05/2017 00:06, Remi Forax wrote: > Hi all, > I wonder if it's an issue or if there is a rational for not suppressing this error when using --ignore-signing-information. > > I'm patching the compiler related modules of the jdk9 with the one from amber/langtools, so i know that i'm doing something borderline but i gently ask jlink to do not bother me with the hash signature. > > ... > jlink --module-path ./target/main/artifact:./deps:/usr/jdk/jdk-9-b167/jmods > --add-modules jdk.policytool,jdk.charsets,jdk.pack,jdk.xml.dom,java.xml,jdk.deploy,java.datatransfer,java.jnlp,java.desktop,com.github.forax.pro.uberbooter,java.se,jdk.snmp,java.security.sasl,jdk.deploy.controlpanel,jdk.zipfs,java.base,jdk.crypto.ec,com.github.forax.pro.main,jdk.management.agent,jdk.jshell,jdk.editpad,oracle.desktop,jdk.jsobject,jdk.unsupported,java.smartcardio,jdk.scripting.nashorn,java.security.jgss,com.github.forax.pro.plugin.convention,jdk.dynalink,java.activation,java.sql,java.logging,jdk.jfr,jdk.internal.vm.ci,com.github.forax.pro.plugin.packager,jdk.packager.services,jdk.net,javafx.controls,jdk.internal.ed,jdk.compiler,jdk.naming.rmi,jdk.jconsole,jdk.internal.le,jdk.packager,jdk.jdwp.agent,jdk.internal.vm.compiler,com.github.forax.pro.daemon.imp,java.scripting,jdk.jartool,java.rmi,jdk.jdi,jdk.javaws,jdk.rmic,com.github.forax.pro.plugin.resolver,jdk.jstatd,jdk.httpserver,java.se.ee,jdk.jcmd,javafx.base,com.github.forax.pro.plugin.linker,com.github.forax.pro.plugin.compiler,jdk.plugin.dom,com.github.forax.pro.plugin.modulefixer,jdk.javadoc,jdk.xml.ws,java.sql.rowset,jdk.sctp,javafx.swing,jdk.jlink,jdk.scripting.nashorn.shell,oracle.net,java.xml.bind,java.compiler,javafx.graphics,javafx.fxml,jdk.plugin,javafx.media,jdk.accessibility,jdk.security.jgss,javafx.web,com.github.forax.pro.plugin.uberpackager,jdk.hotspot.agent,javafx.deploy,java.xml.crypto,jdk.incubator.httpclient,jdk.plugin.server,jdk.crypto.cryptoki,java.naming,java.prefs,jdk.internal.opt,jdk.attach,java.transaction,java.xml.ws,java.xml.ws.annotation,java.management,jdk.xml.bind,jdk.internal.jvmstat,java.instrument,jdk.management,jdk.security.auth,jdk.jdeps,jdk.aot,java.corba,java.management.rmi,jdk.naming.dns,jdk.localedata > --launcher pro=com.github.forax.pro.main/com.github.forax.pro.main.Main > --compress 0 > --ignore-signing-information > --output ./target/image > Error: Hash of jdk.javadoc (5b3a628bce2aad807392ea8a3f2ce69a652e885e33db586694b47fcf6f5ebf52) differs to expected hash (6bdf7f26f72c9f102c5052f8ddf68a80a4b18f997a16e56805e3acc7347469b5) recorded in java.base > `--ignore-signing-information` is used when linking signed modular JARs into a run-time image and is nothing to do with the integrity hashes that are used to avoid mixing tied modules from different builds. If you've patched jdk.compiler, jdk.javadoc, and other modules in your jdk9/dev or jigsaw/jake build then I wouldn't expect any issues. I'm guessing therefore that you've patched the packaged modules or less re-created a subset of the JMOD files. When you do that then the hashes stored in java.base will not match. If you re-create java.base.jmod then it will generate the hashes to match the updated modules. Alternatively for your local testing you could just drop the ModuleHashes class file attribute. -Alan From Alan.Bateman at oracle.com Mon May 8 08:17:18 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 May 2017 09:17:18 +0100 Subject: Reminder to update JVMS In-Reply-To: References: Message-ID: On 06/05/2017 18:13, Stephan Herrmann wrote: > I just happened to search for the specification of the class file > representation of modules. I was quite surprised to find nothing in JVMS. > > Then I found: > "The changes to the class-file chapter in support of module declarations > are not included in this draft of the JVMS; they will be included in > a future version of this specification." > > Makes me wonder, if this is still work in progress? > > Anyway, please don't forget :) Are you able to use the "JPMS: Modules in the Java Language and JVM" (lang-vm.html) document? The second section is "Modules in the JVM" which provides the details on the binary form of a module declaration and also a summary of what will be updated in the JVMS. I checked the lang-vm.html document in the JCP submission and it's the same version of the document (2017/2/23 15:27 -0800?[9d1c4ba194e3]) that is linked from the Project Jigsaw page. The class file changes matches the current EA downloads too. The updated `javap` in JDK 9 is probably useful to you as it can dissemble a module-info.class. Also its `-v` option should allow to quickly look at the Module* attributes and the constant pool. -Alan From sander.mak at luminis.eu Mon May 8 13:25:41 2017 From: sander.mak at luminis.eu (Sander Mak) Date: Mon, 8 May 2017 13:25:41 +0000 Subject: Confusing javac error message Message-ID: When invoking javac (build 9-ea+168-jigsaw-nightly-h6355-20170504) with `--add-exports`, I meant to export sun.util but accidentally typed sun.misc: `javac --add-exports java.base/sun.misc=ALL-UNNAMED `. This produced the following, mildly confusing error: > error: module reads package sun.misc from both jdk.unsupported and java.base First of all, it seems like the phrase 'unnamed' is missing from this error. Furthermore, there's no sun.misc package in java.base (would be nice to get an error pointing that out), so how can we read the package from two modules? Sander From mike.milinkovich at eclipse.org Mon May 8 16:11:58 2017 From: mike.milinkovich at eclipse.org (Mike Milinkovich) Date: Mon, 8 May 2017 12:11:58 -0400 Subject: Views on JSR 376 from the Eclipse JDT team In-Reply-To: <20170506153041.482217656@eggemoggin.niobe.net> References: <590CDF00.9010306@oracle.com> <20170506124944.217983127@eggemoggin.niobe.net> <1DBAB83B-D775-4FA9-95E7-2DA0532143E0@eclipse.org> <20170506153041.482217656@eggemoggin.niobe.net> Message-ID: <32804328-bd45-02c5-6734-80c1045c5eb0@eclipse.org> On 2017-05-06 6:30 PM, mark.reinhold at oracle.com wrote: >> You are comparing apples to oranges. The Eclipse Compiler for Java is >> neither a tool, nor a framework. > A compiler is a particular kind of software-development tool, last > I checked. An important one, certainly, but a tool nonetheless. > Surely there is a distinction between those tools, libraries and frameworks that require porting to run on top of the Java 9 language specification, and a compiler which provides an independent implementation of it? > So: Is it the view of the Eclipse Foundation that any release of the > Java SE Platform that changes the Java programming language must wait > until the maintainers of the Eclipse Compiler for Java are satisfied > with the draft Java Language Specification? It is the view of the Eclipse Foundation that any release of any JCP specification must be sufficiently complete that it enables independent implementations. > If not, then how would you characterize the conditions under which an > update to the JLS may be considered ready for release? I am sure that you are aware that in past releases of the Java SE Platform, the Eclipse team were not completely satisfied with the spec. However, in all past cases they have been willing and able to work through the pain. I believe you've seen Stephan's recent blog post[1] about continuing issues with Java 8, for example. There is no simple answer to your question. It is a matter of degree. What I can say is that this is the first time the ecj team has expressed this level of dissatisfaction. [1] https://objectteams.wordpress.com/2017/04/02/several-languages-java-8/ -- Mike Milinkovich mike.milinkovich at eclipse.org +1.613.220.3223 (mobile) From alex.buckley at oracle.com Mon May 8 19:34:12 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 08 May 2017 12:34:12 -0700 Subject: Reminder to update JVMS In-Reply-To: References: Message-ID: <5910C834.50301@oracle.com> On 5/6/2017 10:13 AM, Stephan Herrmann wrote: > I just happened to search for the specification of the class file > representation of modules. I was quite surprised to find nothing in JVMS. To clarify, you are referring to the Public Review Specification of JSR 376. > Then I found: > "The changes to the class-file chapter in support of module declarations > are not included in this draft of the JVMS; they will be included in > a future version of this specification." That sentence contains a hyperlink to Chapter 2 of the document "JPMS: Modules in the Java Language and JVM". This document is a normative part of the Public Review Specification, so the class file format supported by Java SE 9 is the union of the JVMS9 draft + the document's Chapter 2. As to precisely how Chapter 2 will affect the final JVMS9 in Java SE 9, the document clearly states: "The sections below will contribute to chapter 4 of JVMS9." > Makes me wonder, if this is still work in progress? As Alan noted, the class file changes are stable. Alex From martinrb at google.com Tue May 9 00:45:50 2017 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 May 2017 17:45:50 -0700 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: <06d99b2d-2140-7c22-a8a0-612de7d83b5d@redhat.com> References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <06d99b2d-2140-7c22-a8a0-612de7d83b5d@redhat.com> Message-ID: On Tue, May 2, 2017 at 2:40 AM, Andrew Dinn wrote: > > Just wanted to post a heads-up that this fall-back behaviour has now > been implemented in Byteman release 4.0.0-BETA5. Thanks - I can confirm 4.0.0-BETA5 works for me. From Alan.Bateman at oracle.com Tue May 9 10:56:32 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 May 2017 11:56:32 +0100 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <06d99b2d-2140-7c22-a8a0-612de7d83b5d@redhat.com> Message-ID: On 09/05/2017 01:45, Martin Buchholz wrote: > On Tue, May 2, 2017 at 2:40 AM, Andrew Dinn wrote: > >> Just wanted to post a heads-up that this fall-back behaviour has now >> been implemented in Byteman release 4.0.0-BETA5. > > Thanks - I can confirm 4.0.0-BETA5 works for me. Also Mandy has pushed the fix for JDK-8020801 to jdk9/dev so I assume Andrew will be able to drop the workaround/fall-back in Byteman at some point too. -Alan From adinn at redhat.com Tue May 9 12:36:54 2017 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 9 May 2017 13:36:54 +0100 Subject: Accessing module internals from bytecode rewriting agent In-Reply-To: References: <2099da91-fe6e-a5ed-3396-1e57284a75f1@oracle.com> <06d99b2d-2140-7c22-a8a0-612de7d83b5d@redhat.com> Message-ID: <8c6ac77d-aae1-2fb6-36e2-ba76d450e339@redhat.com> On 09/05/17 11:56, Alan Bateman wrote: > On 09/05/2017 01:45, Martin Buchholz wrote: > >> On Tue, May 2, 2017 at 2:40 AM, Andrew Dinn wrote: >> >>> Just wanted to post a heads-up that this fall-back behaviour has now >>> been implemented in Byteman release 4.0.0-BETA5. >> >> Thanks - I can confirm 4.0.0-BETA5 works for me. > Also Mandy has pushed the fix for JDK-8020801 to jdk9/dev so I assume > Andrew will be able to drop the workaround/fall-back in Byteman at some > point too. Yes, that is on the cards -- but will probably only happen when I /need/ to do a new release for some other reason. Thanks again to Mandy for the rapid fix. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Tue May 9 13:02:34 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 May 2017 14:02:34 +0100 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170504103906.689386446@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> Message-ID: <7450295c-ad02-1be1-1717-724284585243@oracle.com> Just a quick note to say that the initial support for the Automatic-Module-Name JAR file attribute is in the Jigsaw EA builds [1] for those that want to try it out. -Alan [1] http://jdk.java.net/jigsaw/ From stephan.herrmann at berlin.de Tue May 9 14:56:11 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Tue, 9 May 2017 16:56:11 +0200 Subject: An alternative to "restricted keywords" Message-ID: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> (1) I understand the need for avoiding that new module-related keywords conflict with existing code, where these words may be used as identifiers. Moreover, it must be possible for a module declaration to refer to packages or types thusly named. However, (2) The currently proposed "restricted keywords" are not appropriately specified in JLS. (3) The currently proposed "restricted keywords" pose difficulties to the implementation of all tools that need to parse a module declaration. (4) A simple alternative to "restricted keywords" exists, which has not received the attention it deserves. Details: (2) The current specification implicitly violates the assumption that parsing can be performed on the basis of a token stream produced by a scanner (aka lexer). From discussion on this list we learned that the following examples are intended to be syntactically legal: module m { requires transitive transitive; } module m { requires transitive; } (Please for the moment disregard heuristic solutions, while we are investigating whether generally "restricted keywords" is a well-defined concept, or not.) Of the three occurrences of "transitive", #1 is a keyword, the others are identifiers. At the point when the parser has consumed "requires" and now asks about classification of the word "transitive", the scanner cannot possible answer this classification. It can only answer for sure, after the *parser* has accepted the full declaration. Put differently, the parser must consume more tokens than have been classified by the Scanner. Put differently, to faithfully parse arbitrary grammars using a concept of "restricted keywords", scanners must provide speculative answers, which may later need to be revised by backtracking or similar exhaustive exploration of the space of possible interpretations. The specification is totally silent about this fundamental change. (3) "restricted keywords" pose three problems to tool implementations: (3.a) Any known practical approach to implement a parser with "restricted keywords" requires to leverage heuristics, which are based on the exact set of rules defined in the grammar. Such heuristics reduce the look-ahead that needs to be performed by the scanner, in order to avoid the full exhaustive exploration mentioned above. A set of such heuristic is extremely fragile and can easily break when later more rules are added to the grammar. This means small future language changes can easily break any chosen strategy. (3.b) If parsing works for error-free input, this doesn't imply that a parser will be able to give any useful answer for input with syntax errors. As a worst-case example consider an arbitrary input sequence consisting of just the two words "requires" and "transitive" in random order and with no punctuation. A parser will not be able to detect any structure in this sequence. By comparison, normal keywords serve as a baseline, where parsing typically can resume regardless of any leading garbage. While this is not relevant for normal compilation, it is paramount for assistive functions, which most of the time operate on incomplete text, likely to contain even syntax errors. Strictly speaking, any "module declaration" with syntax errors is not a ModuleDeclaration, and thus none of the "restrictive keywords" can be interpreted as keywords (which per JLS can only happen inside a ModuleDeclaration). All this means, that functionality like code completion is systematically broken in a language using "restricted keywords". (3.c) Other IDE functionality assumes that small fragments of the input text can be scanned out of context. The classical example here is syntax highlighting but there are more examples. Any such functionality has to be re-implemented, replacing the highly efficient local scanning with full parsing of the input text. For functionality that is implicitly invoked per keystroke, or on mouse hover etc, this difference in efficiency negatively affects the overall user experience of an IDE. (4) The following proposal avoids all difficulties described above: * open, module, requires, transitive, exports, opens, to, uses, provides, and with are "module words", to which the following interpretation is applied: * within any ordinary compilation unit, a module word is a normal identifier. * within a modular compilation unit, all module words are (unconditional) keywords. * We introduce three new auxiliary non-terminals: LegacyPackageName: LegacyIdentifier LegacyPackageName . LegacyIdentifier LegacyTypeName: LegacyIdentifier LegacyTypeName . LegacyIdentifier LegacyIdentifier: Identifier ^open ^module ... ^with * We modify all productions in 7.7, replacing PackageName with LegacyPackageName and replacing TypeName with LegacyTypeName. * After parsing, each of the words '^open', '^module' etc. is interpreted by removing the leading '^' (escape character). Here, '^' is chosen as the escape character following the precedent of Xtext. Plenty of other options for this purpose are possible, too. This proposal completely satisfies the requirements (1), and avoids all of the problems (2) and (3). There's an obvious price to pay: users will have to add the escape character when referring to code that uses a module word as a package name or type name. Not only is this a very low price compared to the benefits; one can even argue that it also helps the human reader of a module declaration, because it clearly marks which occurrences of a module word are indeed identifiers. An IDE can easily help in interactively adding escapes where necessary. Finally, in this trade-off it is relevant to consider the expected frequencies: legacy names (needing escape) will surely be the exception - by magnitudes. So, the little price needing to be paid, will only affect a comparatively small number of locations. Stephan From stephan.herrmann at berlin.de Tue May 9 17:19:56 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Tue, 9 May 2017 19:19:56 +0200 Subject: bidirectional dependencies in AOP-like languages? Message-ID: Over in jpms-spec-experts I still see discussion about #CyclicDependences, which made me wonder (admittedly late), what will be the solution for AOP-like languages which inherently need bidirectional dependencies? Is it correct that at runtime cycles pose no problems? Let's assume a language using Instrumentation & ClassFileTransformer for weaving an aspect into its base. Lets assume the aspect statically declares a dependence on that base. Now the weaver needs to make the aspect accessible for base-to-aspect calls, i.e., insert a dependency inverse to the existing one. Has anybody tested, whether Instrumentation.redefineModule() works for this kind of use case? Are there any caveats? Better approaches? Is it right that a strategy using compile-time weaving (rather than load time weaving and reflection) will not work in a modular application (due to #CyclicDependences)? thanks, Stephan From rgoers at apache.org Tue May 9 18:51:08 2017 From: rgoers at apache.org (Ralph Goers) Date: Tue, 9 May 2017 11:51:08 -0700 Subject: Need help implementing Java modules Message-ID: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> I am attempting to modularize Log4j and am running into some trouble understanding how this can work. Log4j supports many Appenders, such as the FlumeAppender, KafkaAppender, CassandraAppender, and JPAAppender. Each of these use their respective libraries to cause the log events to be delivered to their destinations. All of these frameworks perform logging and so would have a dependency on a logging framework. For example, the FlumeAppender passes the LogEvent to an embedded Flume agent that then sends the event to a remote Flume node. The embedded agent performs logging using the Log4j 1 API and also uses other libraries that perform logging. So if Flume was modularized it would require some logging module. That module would almost certainly route logging calls back to Log4j 2 and so would require Log4j 2 and result in a circularity. In the same manner, the KafkaAppender uses the SLF4J API. When the implementation of that is log4j-slf4j-impl a circularity will occur. Since the Java module system forbids circularities I don?t understand how Log4j (or Logback) can realistically be modularized. We also noticed that System.LoggerFinder [1] now provides a new way to integrate an external logging framework with Java. If Log4j creates the service provider for that how will that not cause a runtime circularity since java.base now is bound to log4j-api and/or log4j-core and both of those require java.base. Ralph 1. http://docs.jboss.org/hibernate/orm/4.3/topical/html/logging/Logging.html From blackdrag at gmx.org Tue May 9 19:50:36 2017 From: blackdrag at gmx.org (Jochen Theodorou) Date: Tue, 9 May 2017 21:50:36 +0200 Subject: Need help implementing Java modules In-Reply-To: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> Message-ID: <59121D8C.1070101@gmx.org> On 09.05.2017 20:51, Ralph Goers wrote: > I am attempting to modularize Log4j and am running into some trouble > understanding how this can work. > > Log4j supports many Appenders, such as the FlumeAppender, > KafkaAppender, CassandraAppender, and JPAAppender. Each of these use > their respective libraries to cause the log events to be delivered to > their destinations. All of these frameworks perform logging and so > would have a dependency on a logging framework. For example, the > FlumeAppender passes the LogEvent to an embedded Flume agent that > then sends the event to a remote Flume node. The embedded agent > performs logging using the Log4j 1 API and also uses other libraries > that perform logging. So if Flume was modularized it would require > some logging module. That module would almost certainly route logging > calls back to Log4j 2 and so would require Log4j 2 and result in a > circularity. In the same manner, the KafkaAppender uses the SLF4J > API. When the implementation of that is log4j-slf4j-impl a > circularity will occur. Since the Java module system forbids > circularities I don?t understand how Log4j (or Logback) can > realistically be modularized. I will try to explain and depend on others to correct me if I am wrong.. Assuming you have your log4j-base module, which will also provide an interface to communicate with a implementation of log4j. A FlumeAppender would then for example depends on that interface, but log4j-base would not depend on the Flume Appender. So the question is how log4j-base can then get the FlumeAppender... which is what SPI is for. So every Appender would then be a service provider. > We also noticed that System.LoggerFinder [1] now provides a new way > to integrate an external logging framework with Java. If Log4j > creates the service provider for that how will that not cause a > runtime circularity since java.base now is bound to log4j-api and/or > log4j-core and both of those require java.base. > > Ralph > > 1. > http://docs.jboss.org/hibernate/orm/4.3/topical/html/logging/Logging.html > SPI again. Since Log4j depends on java.base for compilation, but java.base does still compile without log4j-api or -core it is not a circular dependency. bye Jochen From alex.buckley at oracle.com Tue May 9 19:54:53 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 09 May 2017 12:54:53 -0700 Subject: Need help implementing Java modules In-Reply-To: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> Message-ID: <59121E8D.4050906@oracle.com> On 5/9/2017 11:51 AM, Ralph Goers wrote: > I am attempting to modularize Log4j and am running into some trouble > understanding how this can work. > > Log4j supports many Appenders, such as the FlumeAppender, > KafkaAppender, CassandraAppender, and JPAAppender. I don't know how Log4j is currently discovering Appenders, but I guess it's through reflective means (Class.forName, classpath scanning, etc). Services are a better approach because then the JDK does the discovery while you focus on the type-safe protocol between Log4j and Appenders. Consider this: - The Log4j module should not require the FlumeAppender module et al. - The Log4j module should export a service interface AppenderIntf. - The FlumeAppender module should i) require the Log4j module to gain access to AppenderIntf, and ii) provide its Flume-specific implementation of the service interface ('provides AppenderIntf with FlumeAppenderImpl;'). If the FlumeAppender module requires helper modules, and they in turn require the Log4j module, that's fine. - The Log4j module should use its own service interface ('uses AppenderIntf;') and discover appender implementations via ServiceLoader. Alex From novanic at gmx.de Tue May 9 20:33:12 2017 From: novanic at gmx.de (Sven Strohschein) Date: Tue, 9 May 2017 22:33:12 +0200 Subject: Problems with non-exported split packages (for resources/files) Message-ID: Hi, I'm trying to switch a project from OSGi to Jigsaw with the latest JDK 9 ea build (168). The most is working fine and I like that it is much easier and much more practical than OSGi. But there is one strong limitation within Jigsaw which I don't understand, which is much more restrictive even than OSGi and which will be a big problem to introduce Jigsaw in other projects: Non-exported split packages (http://openjdk.java.net/projects/jigsaw/spec/issues/#AvoidConcealedPackageConflicts). OSGi allows such private split packages, Jigsaw doesn't. In my understanding only packages which are declared in the module-info class should get checked for split packages and Jigsaw shouldn't care about internal module structures. Mark Reinhold wrote, that it will be solved in a future release, but this is a strong, useless, unnecessary restrictive limitation which will cause a lot of problems when the people try to use Jigsaw in their projects. It blocks me and it will block others. And even worse: It will lead and force to senseless internal module structures, which can get reverted when the future release gets released... There is a very practical example which isn't a bad practice: Resources. Imagine that every Java module has its own ResourceBundle properties for multi-language support (or imagine any other file which is not a Java class). These files are typically placed in a "conf", "etc" or "resources" folder structure and added to the classpath. It should not be necessary that every module has another place for such resources, that would be a bad solution! Java throws a LayerInstantiationException on starting the application when multiple modules have the same resource folder structures... How should resources be organized in your opinion? Should conf/etc/resources renamed and restructured to common package names starting with the top level domain? I don't want to change my projects in such a way for Jigsaw, it shouldn't be necessary... I would be happy when this is fixed within the first release of Java 9 / Jigsaw (and I don't belong to Red Hat or IBM ;-)). Thank you in advance. Best regards, Sven Strohschein From rgoers at apache.org Tue May 9 22:04:51 2017 From: rgoers at apache.org (Ralph Goers) Date: Tue, 9 May 2017 15:04:51 -0700 Subject: Need help implementing Java modules In-Reply-To: <59121D8C.1070101@gmx.org> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> Message-ID: > On May 9, 2017, at 12:50 PM, Jochen Theodorou wrote: > > On 09.05.2017 20:51, Ralph Goers wrote: > >> We also noticed that System.LoggerFinder [1] now provides a new way >> to integrate an external logging framework with Java. If Log4j >> creates the service provider for that how will that not cause a >> runtime circularity since java.base now is bound to log4j-api and/or >> log4j-core and both of those require java.base. >> >> Ralph >> >> 1. >> http://docs.jboss.org/hibernate/orm/4.3/topical/html/logging/Logging.html >> > > > SPI again. Since Log4j depends on java.base for compilation, but java.base does still compile without log4j-api or -core it is not a circular dependency. > > bye Jochen Pardon me for being dense, but my reading said that Java modules disallowed runtime cycles as well as compile time. Once LoggerFinder binds with the module that provides that service does that not create a runtime dependency? I don?t recall seeing anything describing what the behavior of that is. Ralph From mandy.chung at oracle.com Tue May 9 22:22:12 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 9 May 2017 15:22:12 -0700 Subject: Problems with non-exported split packages (for resources/files) In-Reply-To: References: Message-ID: On May 9, 2017, at 1:33 PM, Sven Strohschein wrote: > > There is a very practical example which isn't a bad practice: Resources. Imagine that every Java module has its own ResourceBundle properties for multi-language support (or imagine any other file which is not a Java class). These files are typically placed in a "conf", "etc" or "resources" folder structure and added to the classpath. It should not be necessary that every module has another place for such resources, that would be a bad solution! Can you point me to some example library that puts ResourceBundles in these directories? The example resource bundles I looked at are typically local and private to a library/application in its own package. In those cases, the resource bundles are encapsulated. A module can load a resource bundle from another module if it?s open to it for access, in the same fashion as resources. When the localized resource bundles must be distributed in multiple JAR files, there are several options: 1. leaving those JAR files on classpath will continue to work. 2. loading as automatic modules on modulepath will work if the package has only .properties file 3. migrate to service provider in modules Migrating from unnamed module to named module, it requires work to eliminate the split package [1]. > Java throws a LayerInstantiationException on starting the application when multiple modules have the same resource folder structures... How should resources be organized in your opinion? Should conf/etc/resources renamed and restructured to common package names starting with the top level domain? One option is to move resources to META-INF directory and other directory whose name is not a valid package name [2]. > I don't want to change my projects in such a way for Jigsaw, it shouldn't be necessary? > Your projects should run fine if you run with -classpath as in JDK 8 unless you depend on JDK internal APIs. Have you tried that? Mandy [1] http://download.java.net/java/jdk9/docs/api/java/util/ResourceBundle.html#bundleprovider [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/000392.html From alex.buckley at oracle.com Tue May 9 22:39:14 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 09 May 2017 15:39:14 -0700 Subject: Need help implementing Java modules In-Reply-To: References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> Message-ID: <59124512.2010505@oracle.com> On 5/9/2017 3:04 PM, Ralph Goers wrote: > Pardon me for being dense, but my reading said that Java modules > disallowed runtime cycles as well as compile time. Once LoggerFinder > binds with the module that provides that service does that not create > a runtime dependency? I don?t recall seeing anything describing what > the behavior of that is. The module system disallows cycles in the 'requires' directives of module declarations. The module system allows cycles in the "reads" relation of run-time modules. When java.base 'uses LoggerFinder;' and invokes ServiceLoader to find providers, there is indeed a "reads" cycle created between the provider module and java.base. ServiceLoader is not special in creating this cycle -- you can create them yourself with the two addReads methods in the API, and all automatic modules have cyclic readability. But there is no 'requires' cycle. Alex From rgoers at apache.org Wed May 10 00:20:31 2017 From: rgoers at apache.org (Ralph Goers) Date: Tue, 9 May 2017 17:20:31 -0700 Subject: Need help implementing Java modules In-Reply-To: <59124512.2010505@oracle.com> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> Message-ID: <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> > On May 9, 2017, at 3:39 PM, Alex Buckley wrote: > > On 5/9/2017 3:04 PM, Ralph Goers wrote: >> Pardon me for being dense, but my reading said that Java modules >> disallowed runtime cycles as well as compile time. Once LoggerFinder >> binds with the module that provides that service does that not create >> a runtime dependency? I don?t recall seeing anything describing what >> the behavior of that is. > > The module system disallows cycles in the 'requires' directives of module declarations. The module system allows cycles in the "reads" relation of run-time modules. > > When java.base 'uses LoggerFinder;' and invokes ServiceLoader to find providers, there is indeed a "reads" cycle created between the provider module and java.base. ServiceLoader is not special in creating this cycle -- you can create them yourself with the two addReads methods in the API, and all automatic modules have cyclic readability. But there is no 'requires' cycle. > > Alex > Thanks Alex, that would indeed be enough to break the cycle as log4j-api can easily use serviceLoader to bind with log4j-core as that is a very minor change. However your previous response leads me to another question. Log4j already has a robust plugin approach for users to implement their own Appenders, Filters, Layouts, and other Log4j components. We are not going to modify that as it would severely impact our users who have already implemented custom components and what we have works very well. Although the FlumeAppender I mentioned previously is provided by Log4j it, and all other Log4j components, are located via an annotation processor provided by Log4j. The processor runs at compile time and generates a file named META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat. All of these files are then located at runtime using ClassLoader.getResources(). As we parse the configuration provided by the user we instantiate the plugins using the class names provided in those files and the static builder or static factory method provided in the plugin class. From what I am reading in the javadoc we should not have any trouble with this because META-INF is not a valid package name so the resource file will not be hidden. Log4j uses reflection to call the static builder or static method to create the plugin. With all this in mind, if users create modules will they be required to declare the packages where they have created plugins as ?open? to log4j for this to work? I am assuming that Log4j will be able to access this module without having to declare a dependency on it? Thanks, Ralph From kevin.rushforth at oracle.com Wed May 10 01:19:55 2017 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 09 May 2017 18:19:55 -0700 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <1f806cdc-a241-04aa-6e4b-0b11abb1f0a9@gmail.com> References: <5907D11D.20609@oracle.com> <1f806cdc-a241-04aa-6e4b-0b11abb1f0a9@gmail.com> Message-ID: <59126ABB.8090304@oracle.com> inline Peter Levart wrote: > Hi Kevin, > > On 05/02/2017 02:21 AM, Kevin Rushforth wrote: >> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >> >> Please review the proposed fix for: >> >> https://bugs.openjdk.java.net/browse/JDK-8177566 >> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >> >> Details of the fix as well as notes to reviewers are in the bug >> report [1] (e.g., I've also generated separate webrevs for the fix >> itself, the doc changes, and the test changes). >> >> -- Kevin >> >> [1] >> https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 >> > > I think it is very unusual to handle unqualified exports as something > special, different from qualified exports. I know what the reasoning > is: if a package is exported unconditionally (to everyone) then it is > part of public API and so the trampoline may access members of that > package on behalf of anyone. But such reasoning is just a consequence > of the lack of a finer-grained (per-module) access support in JavaFX. > I know it is too much to ask for JDK 9, but could JavaFX in say JDK > 10, given current API, somehow determine on whose behalf it is making > the trampoline access? If it could, then the trampoline could allow > qualified exports to be effective too. The primary remedy is for the application to use a qualified "opens" to the appropriate javafx module. For example, to allow the JavaBeanXXXProperty classes the ability to access a class in your module, your module needs to "opens my.package to javafx.base". The only difference between what you propose and what was implemented is qualified exports versus qualified opens, which really shouldn't be too much of an issue for applications (such applications already need to use qualified opens to allow access to their FXML controller class). The only reason we mention unconditional exports as an alternative is for the benefit of application that happen to already have their package exported unconditionally. -- Kevin > Regards, Peter > From forax at univ-mlv.fr Wed May 10 06:50:15 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 10 May 2017 06:50:15 +0000 Subject: Need help implementing Java modules In-Reply-To: <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> Message-ID: <1D760ED4-8375-4ADE-A7C1-51589E73B329@univ-mlv.fr> On May 10, 2017 2:20:31 AM GMT+02:00, Ralph Goers wrote: > >> On May 9, 2017, at 3:39 PM, Alex Buckley >wrote: >> >> On 5/9/2017 3:04 PM, Ralph Goers wrote: >>> Pardon me for being dense, but my reading said that Java modules >>> disallowed runtime cycles as well as compile time. Once LoggerFinder >>> binds with the module that provides that service does that not >create >>> a runtime dependency? I don?t recall seeing anything describing >what >>> the behavior of that is. >> >> The module system disallows cycles in the 'requires' directives of >module declarations. The module system allows cycles in the "reads" >relation of run-time modules. >> >> When java.base 'uses LoggerFinder;' and invokes ServiceLoader to find >providers, there is indeed a "reads" cycle created between the provider >module and java.base. ServiceLoader is not special in creating this >cycle -- you can create them yourself with the two addReads methods in >the API, and all automatic modules have cyclic readability. But there >is no 'requires' cycle. >> >> Alex >> > >Thanks Alex, that would indeed be enough to break the cycle as >log4j-api can easily use serviceLoader to bind with log4j-core as that >is a very minor change. However your previous response leads me to >another question. > >Log4j already has a robust plugin approach for users to implement their >own Appenders, Filters, Layouts, and other Log4j components. We are not >going to modify that as it would severely impact our users who have >already implemented custom components and what we have works very well. >Although the FlumeAppender I mentioned previously is provided by Log4j >it, and all other Log4j components, are located via an annotation >processor provided by Log4j. The processor runs at compile time and >generates a file named >META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat. >All of these files are then located at runtime using >ClassLoader.getResources(). As we parse the configuration provided by >the user we instantiate the plugins using the class names provided in >those files and the static builder or static factory method provided in >the plugin class. From what I am reading in the javadoc we should not >have any trouble with this because META-INF is not a valid package name >so the resource file will not be hidden. Log4j uses reflection to call >the static builder or static method to create the plugin. > >With all this in mind, if users create modules will they be required to >declare the packages where they have created plugins as ?open? to log4j >for this to work? It depends if when you call the static method you need to bypass the encapsulation or not i.e. if your current code uses setAccessible, yes, the plugin's module has to be opened. In any case, you need to add a read edge at runtime from log4j to the plugin, otherwise you will not find the plugins class. > I am assuming that Log4j will be able to access this >module without having to declare a dependency on it? yes, if you add a read edge at runtime. > >Thanks, >Ralph cheers, R?mi -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From Alan.Bateman at oracle.com Wed May 10 07:24:41 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 May 2017 08:24:41 +0100 Subject: bidirectional dependencies in AOP-like languages? In-Reply-To: References: Message-ID: <75cbfb10-887d-e4da-c327-b1af7b571ef8@oracle.com> On 09/05/2017 18:19, Stephan Herrmann wrote: > : > > Let's assume a language using Instrumentation & ClassFileTransformer for > weaving an aspect into its base. > Lets assume the aspect statically declares a dependence on that base. > Now the weaver needs to make the aspect accessible for base-to-aspect > calls, > i.e., insert a dependency inverse to the existing one. > > Has anybody tested, whether Instrumentation.redefineModule() works for > this kind of use case? Are there any caveats? Better approaches? If the byte code weaving is done at class load time or dynamically (retransformClasses/redefineClasses) then you should be able to get this to work, assuming the readability and visibility supports it. I can't quite tell if "base" means "java.base". Assuming it is, then you should be able to instrument code in java.base with references to classes that you have on the boot class path. As part of the compatibility support to keep existing agents working, the VM will update java.base to read the unnamed module of the boot loader when you modify a class in java.base. Classes in java.base are defined to the boot loader, as is code on the boot class path, so visibility is not an issue either. More on this in the "Instrumenting code in modules" and "Visibility" sections of the java.lang.instrument docs [1]. If you doing super advanced stuff then you may need to use redefineModule to provide additionally readability but you shouldn't have any issues there. More likely, visibility will the change when you have cycles but if you have that working now then it should continue to work. > > Is it right that a strategy using compile-time weaving > (rather than load time weaving and reflection) > will not work in a modular application (due to #CyclicDependences)? > Can you expand a bit on this? Is the user code and aspect compiled to different modules with static references to each other? -Alan [1] http://download.java.net/java/jigsaw/docs/api/java/lang/instrument/package-summary.html From alessiostalla at gmail.com Wed May 10 07:48:18 2017 From: alessiostalla at gmail.com (Alessio Stalla) Date: Wed, 10 May 2017 09:48:18 +0200 Subject: Problems with non-exported split packages (for resources/files) In-Reply-To: References: Message-ID: Case in point, the rapid application development framework Portofino [1] is split in several (non-Jigsaw) modules each with its message bundle fragment, which is named portofino-messages and placed in the empty package. Also, each module is defined by a class which has to be in a well-defined package, the same for all modules, but this is more of an implementation detail. I'm not saying that these are ideal design choices, but that these exist in the wild. [1] https://github.com/ManyDesigns/Portofino On 10 May 2017 at 00:22, Mandy Chung wrote: > On May 9, 2017, at 1:33 PM, Sven Strohschein wrote: > > > > There is a very practical example which isn't a bad practice: Resources. > Imagine that every Java module has its own ResourceBundle properties for > multi-language support (or imagine any other file which is not a Java > class). These files are typically placed in a "conf", "etc" or "resources" > folder structure and added to the classpath. It should not be necessary > that every module has another place for such resources, that would be a bad > solution! > > Can you point me to some example library that puts ResourceBundles in > these directories? > > The example resource bundles I looked at are typically local and private > to a library/application in its own package. In those cases, the resource > bundles are encapsulated. A module can load a resource bundle from another > module if it?s open to it for access, in the same fashion as resources. > > When the localized resource bundles must be distributed in multiple JAR > files, there are several options: > 1. leaving those JAR files on classpath will continue to work. > 2. loading as automatic modules on modulepath will work if the package has > only .properties file > 3. migrate to service provider in modules > Migrating from unnamed module to named module, it requires work to > eliminate the split package [1]. > > > > Java throws a LayerInstantiationException on starting the application > when multiple modules have the same resource folder structures... How > should resources be organized in your opinion? Should conf/etc/resources > renamed and restructured to common package names starting with the top > level domain? > > One option is to move resources to META-INF directory and other directory > whose name is not a valid package name [2]. > > > I don't want to change my projects in such a way for Jigsaw, it > shouldn't be necessary? > > > > Your projects should run fine if you run with -classpath as in JDK 8 > unless you depend on JDK internal APIs. Have you tried that? > > Mandy > [1] http://download.java.net/java/jdk9/docs/api/java/util/ > ResourceBundle.html#bundleprovider > [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/ > 2016-September/000392.html From Alan.Bateman at oracle.com Wed May 10 13:43:00 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 May 2017 14:43:00 +0100 Subject: Problems with non-exported split packages (for resources/files) In-Reply-To: References: Message-ID: <1463a5b9-ac38-c2d4-9463-56a9948c9aac@oracle.com> On 10/05/2017 08:48, Alessio Stalla wrote: > Case in point, the rapid application development framework Portofino [1] is > split in several (non-Jigsaw) modules each with its message bundle > fragment, which is named portofino-messages and placed in the empty package. By "the empty package" then I assume you mean the resources are in the top-level directory of the JAR file. If that is so then you shouldn't have any issues as there is no encapsulation of resources in the top-level directory. On the other hand, if the resources are compiled, meaning you have compiled the resources to .class files, then there will be an issue because all types in a named module must be in a named package. -Alan From peter.levart at gmail.com Wed May 10 14:32:52 2017 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 10 May 2017 16:32:52 +0200 Subject: Problems with non-exported split packages (for resources/files) In-Reply-To: References: Message-ID: Hi, Two modules can have packages with same name if they are not exported *and* each of the modules loads classes using distinct class loaders, right? This is a limitation imposed primarily because of ability to load multiple modules using same class loader. But if the modules that have types loaded by the same class loader contain concealed (non-exported) packages of the same name and those packages contain only resources, then in theory, JPMS could allow such configuration, because for loading private resources, there's no need to consult the ClassLoader. Each module could load its private resources with the sole help of its ModuleReader. Am I right? Regards, Peter On 05/10/2017 12:22 AM, Mandy Chung wrote: > On May 9, 2017, at 1:33 PM, Sven Strohschein wrote: >> There is a very practical example which isn't a bad practice: Resources. Imagine that every Java module has its own ResourceBundle properties for multi-language support (or imagine any other file which is not a Java class). These files are typically placed in a "conf", "etc" or "resources" folder structure and added to the classpath. It should not be necessary that every module has another place for such resources, that would be a bad solution! > Can you point me to some example library that puts ResourceBundles in these directories? > > The example resource bundles I looked at are typically local and private to a library/application in its own package. In those cases, the resource bundles are encapsulated. A module can load a resource bundle from another module if it?s open to it for access, in the same fashion as resources. > > When the localized resource bundles must be distributed in multiple JAR files, there are several options: > 1. leaving those JAR files on classpath will continue to work. > 2. loading as automatic modules on modulepath will work if the package has only .properties file > 3. migrate to service provider in modules > Migrating from unnamed module to named module, it requires work to eliminate the split package [1]. > > >> Java throws a LayerInstantiationException on starting the application when multiple modules have the same resource folder structures... How should resources be organized in your opinion? Should conf/etc/resources renamed and restructured to common package names starting with the top level domain? > One option is to move resources to META-INF directory and other directory whose name is not a valid package name [2]. > >> I don't want to change my projects in such a way for Jigsaw, it shouldn't be necessary? >> > Your projects should run fine if you run with -classpath as in JDK 8 unless you depend on JDK internal APIs. Have you tried that? > > Mandy > [1] http://download.java.net/java/jdk9/docs/api/java/util/ResourceBundle.html#bundleprovider > [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/000392.html From rgoers at apache.org Wed May 10 13:38:39 2017 From: rgoers at apache.org (Ralph Goers) Date: Wed, 10 May 2017 06:38:39 -0700 Subject: Need help implementing Java modules In-Reply-To: <1D760ED4-8375-4ADE-A7C1-51589E73B329@univ-mlv.fr> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> <1D760ED4-8375-4ADE-A7C1-51589E73B329@univ-mlv.fr> Message-ID: <5E0CD883-6520-477F-AA1F-75A2F089F2F3@apache.org> > On May 9, 2017, at 11:50 PM, Remi Forax wrote: > > > > On May 10, 2017 2:20:31 AM GMT+02:00, Ralph Goers > wrote: >> >> >> With all this in mind, if users create modules will they be required to >> declare the packages where they have created plugins as ?open? to log4j >> for this to work? > > It depends if when you call the static method you need to bypass the encapsulation or not i.e. if your current code uses setAccessible, yes, the plugin's module has to be opened. > > In any case, you need to add a read edge at runtime from log4j to the plugin, otherwise you will not find the plugins class. > >> I am assuming that Log4j will be able to access this >> module without having to declare a dependency on it? > > yes, if you add a read edge at runtime. > >> >> Thanks, >> Ralph > > cheers, > R?mi If log4j-core is an automatic module is it still required to call addReads()? I would think not since it isn?t really a module. Would the module with the plugin still have to declare its package as open? It is almost certain that many of our jars will have to be automatic modules at least to start because none of the dependencies we use have indicated that they are going to support JPMS yet (no great surprise there since the spec isn?t final). Ralph From Alan.Bateman at oracle.com Wed May 10 14:51:09 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 May 2017 15:51:09 +0100 Subject: Problems with non-exported split packages (for resources/files) In-Reply-To: References: Message-ID: On 10/05/2017 15:32, Peter Levart wrote: > Hi, > > Two modules can have packages with same name if they are not exported > *and* each of the modules loads classes using distinct class loaders, > right? That's right. If the modules are defined to different class loaders then they can even export the same API packages as long as one doesn't read the other. > > This is a limitation imposed primarily because of ability to load > multiple modules using same class loader. But if the modules that have > types loaded by the same class loader contain concealed (non-exported) > packages of the same name and those packages contain only resources, > then in theory, JPMS could allow such configuration, because for > loading private resources, there's no need to consult the ClassLoader. > Each module could load its private resources with the sole help of its > ModuleReader. Am I right? If the resources don't need to be encapsulated then the module doesn't need to contain the "package". The packages in an automatic module only contain the packages containing classes so this is why it works with automatic modules. The class loaders that locate resources in modules do so with a ModuleReader. -Alan From alex.buckley at oracle.com Wed May 10 18:44:18 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 10 May 2017 11:44:18 -0700 Subject: Need help implementing Java modules In-Reply-To: <1D760ED4-8375-4ADE-A7C1-51589E73B329@univ-mlv.fr> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> <1D760ED4-8375-4ADE-A7C1-51589E73B329@univ-mlv.fr> Message-ID: <59135F82.6090606@oracle.com> On 5/9/2017 11:50 PM, Remi Forax wrote: > On May 10, 2017 2:20:31 AM GMT+02:00, Ralph Goers > wrote: >> With all this in mind, if users create modules will they be >> required to declare the packages where they have created plugins as >> ?open? to log4j for this to work? > > It depends if when you call the static method you need to bypass the > encapsulation or not i.e. if your current code uses setAccessible, > yes, the plugin's module has to be opened. Yes; I'll talk about that in a parallel mail. > In any case, you need to add a read edge at runtime from log4j to the > plugin, otherwise you will not find the plugins class. No need for a reads edge. If you're using Core Reflection to instantiate classes and access their members, then you're subject to class loader visibility (which is the same as in JDK 8) and module accessibility (hence the need to open or export the plugin package), but readability comes for free. >> I am assuming that Log4j will be able to access this module without >> having to declare a dependency on it? > > yes, if you add a read edge at runtime. No need for that. Alex From alex.buckley at oracle.com Wed May 10 18:58:57 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 10 May 2017 11:58:57 -0700 Subject: Need help implementing Java modules In-Reply-To: <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> Message-ID: <591362F1.9000401@oracle.com> On 5/9/2017 5:20 PM, Ralph Goers wrote: > Log4j already has a robust plugin approach for users to implement > their own Appenders, Filters, Layouts, and other Log4j components. We > are not going to modify that as it would severely impact our users > who have already implemented custom components and what we have works > very well. Although the FlumeAppender I mentioned previously is > provided by Log4j it, and all other Log4j components, are located via > an annotation processor provided by Log4j. The processor runs at > compile time and generates a file named > META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat. > All of these files are then located at runtime using > ClassLoader.getResources(). As we parse the configuration provided by > the user we instantiate the plugins using the class names provided in > those files and the static builder or static factory method provided > in the plugin class. From what I am reading in the javadoc we should > not have any trouble with this because META-INF is not a valid > package name so the resource file will not be hidden. Log4j uses > reflection to call the static builder or static method to create the > plugin. > > With all this in mind, if users create modules will they be required > to declare the packages where they have created plugins as ?open? to > log4j for this to work? I am assuming that Log4j will be able to > access this module without having to declare a dependency on it? Basically you've got it. - The Log4j module does not need to 'requires' any user module, nor invoke addReads to "read" any user module. - Log4j code can continue to invoke Class::forName to obtain a Class object for a class in a user module. This is based purely on class loader visibility, no changes from the module system. - Log4j code can continue to invoke Class::getMethod et al to obtain a Method object for a static builder/factory method. There are no access control rules when simply "inspecting" the Class object via getMethod, getMethods, getFields, etc. - Log4j code can attempt to invoke Method::invoke on its preferred Method object, but for the attempt to succeed, the user module must open or export the class's package. If you mandate that a plugin class must be public with a public builder/factory method, then exporting the package will be enough; otherwise the package needs to be open. - I recommend that you recommend that users should open the package to specifically Log4j, rather than opening the package to everyone (or opening the entire module). You said that your JARs will probably be automatic modules for some time, so you can use the Automatic-Module-name manifest entry to introduce a stable name that user modules can open to. Alex From greggwon at cox.net Wed May 10 20:00:11 2017 From: greggwon at cox.net (Gregg Wonderly) Date: Wed, 10 May 2017 15:00:11 -0500 Subject: Need help implementing Java modules In-Reply-To: References: <5EE95CAD-3E3F-43C4-9400-E9B04B20D211@apache.org> <59121D8C.1070101@gmx.org> <59124512.2010505@oracle.com> <58EE72F0-F6A3-4087-9912-6287E6838AFC@apache.org> Message-ID: <8437CA14-F567-420F-9413-93F8DC1E4492@cox.net> > On May 10, 2017, at 1:58 PM, Alex Buckley wrote: > > On 5/9/2017 5:20 PM, Ralph Goers wrote: >> Log4j already has a robust plugin approach for users to implement >> their own Appenders, Filters, Layouts, and other Log4j components. We >> are not going to modify that as it would severely impact our users >> who have already implemented custom components and what we have works >> very well. Although the FlumeAppender I mentioned previously is >> provided by Log4j it, and all other Log4j components, are located via >> an annotation processor provided by Log4j. The processor runs at >> compile time and generates a file named >> META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat. >> All of these files are then located at runtime using >> ClassLoader.getResources(). As we parse the configuration provided by >> the user we instantiate the plugins using the class names provided in >> those files and the static builder or static factory method provided >> in the plugin class. From what I am reading in the javadoc we should >> not have any trouble with this because META-INF is not a valid >> package name so the resource file will not be hidden. Log4j uses >> reflection to call the static builder or static method to create the >> plugin. >> >> With all this in mind, if users create modules will they be required >> to declare the packages where they have created plugins as ?open? to >> log4j for this to work? I am assuming that Log4j will be able to >> access this module without having to declare a dependency on it? > > Basically you've got it. > > - The Log4j module does not need to 'requires' any user module, nor invoke addReads to "read" any user module. > > - Log4j code can continue to invoke Class::forName to obtain a Class object for a class in a user module. This is based purely on class loader visibility, no changes from the module system. > > - Log4j code can continue to invoke Class::getMethod et al to obtain a Method object for a static builder/factory method. There are no access control rules when simply "inspecting" the Class object via getMethod, getMethods, getFields, etc. > > - Log4j code can attempt to invoke Method::invoke on its preferred Method object, but for the attempt to succeed, the user module must open or export the class's package. If you mandate that a plugin class must be public with a public builder/factory method, then exporting the package will be enough; otherwise the package needs to be open. > > - I recommend that you recommend that users should open the package to specifically Log4j, rather than opening the package to everyone (or opening the entire module). You said that your JARs will probably be automatic modules for some time, so you can use the Automatic-Module-name manifest entry to introduce a stable name that user modules can open to. Alex, what will happen to software that is not log4j that wants to use these Appenders? For something as generic as this, isn?t it really better to just open to everyone? It just seems more and more like the JDK teams focus on locking people out of the JDK is driving everything in the direction of not sharing code for reuse in most places. There are very view directed graphs of dependencies in open source code that really need to be anal about what is open. Software implementing an open/public interface should always be open to all uses of that implementation it seems to me. Gregg From paul.bakker.nl at gmail.com Thu May 11 06:22:11 2017 From: paul.bakker.nl at gmail.com (Paul Bakker) Date: Wed, 10 May 2017 23:22:11 -0700 Subject: #AddExportsInManifest Message-ID: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> Hi all, In #AddExportsInManifest two new manifest entries are proposed. I can only find the following email containing more details: http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-September/000547.html . Is this still current? I'm trying this in an example, but it doesn't seem to have any effect. Also, is the Add-Exports-Private replaced by Add-Opens? The proposal only speaks about exporting to unnamed modules. Is it also possible to export to other modules? E.g. 'Add-Exports: m/p=othermodule? Thanks, Paul Bakker From Alan.Bateman at oracle.com Thu May 11 06:39:23 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 May 2017 07:39:23 +0100 Subject: #AddExportsInManifest In-Reply-To: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> Message-ID: <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> On 11/05/2017 07:22, Paul Bakker wrote: > Hi all, > > In #AddExportsInManifest two new manifest entries are proposed. I can only find the following email containing more details: http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-September/000547.html . > Is this still current? I'm trying this in an example, but it doesn't seem to have any effect. Also, is the Add-Exports-Private replaced by Add-Opens? > > The proposal only speaks about exporting to unnamed modules. Is it also possible to export to other modules? E.g. 'Add-Exports: m/p=othermodule? > The attributes names are "Add-Exports" and "Add-Opens". JEP 261 needs a big update and one of the things needed is to document these attributes in the JEP (as they are JDK-specific attributes rather than attributes for the JAR file spec). As the attributes are for executable JAR files only then the effect is to export or opens the packages to unnamed modules only (so no `=` in the value). -Alan From paul.bakker.nl at gmail.com Thu May 11 06:51:14 2017 From: paul.bakker.nl at gmail.com (Paul Bakker) Date: Wed, 10 May 2017 23:51:14 -0700 Subject: #AddExportsInManifest In-Reply-To: <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> Message-ID: <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> Hi Alan, What is the reason only exports/opens to unnamed are possible? Also, are these implemented in the current jigsaw prototype? I'm having trouble getting it to work, but that might be entirely my own doing... Thanks, Paul > On May 10, 2017, at 11:39 PM, Alan Bateman wrote: > > On 11/05/2017 07:22, Paul Bakker wrote: > >> Hi all, >> >> In #AddExportsInManifest two new manifest entries are proposed. I can only find the following email containing more details: http://mail.openjdk.java.net/pipermail/jpms-spec-observers/2016-September/000547.html . >> Is this still current? I'm trying this in an example, but it doesn't seem to have any effect. Also, is the Add-Exports-Private replaced by Add-Opens? >> >> The proposal only speaks about exporting to unnamed modules. Is it also possible to export to other modules? E.g. 'Add-Exports: m/p=othermodule? >> > The attributes names are "Add-Exports" and "Add-Opens". JEP 261 needs a big update and one of the things needed is to document these attributes in the JEP (as they are JDK-specific attributes rather than attributes for the JAR file spec). > > As the attributes are for executable JAR files only then the effect is to export or opens the packages to unnamed modules only (so no `=` in the value). > > -Alan From Alan.Bateman at oracle.com Thu May 11 07:10:36 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 May 2017 08:10:36 +0100 Subject: #AddExportsInManifest In-Reply-To: <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> Message-ID: On 11/05/2017 07:51, Paul Bakker wrote: > Hi Alan, > > What is the reason only exports/opens to unnamed are possible? > Also, are these implemented in the current jigsaw prototype? I'm having trouble getting it to work, but that might be entirely my own doing... > The attributes are defined for the main application JAR when run with `java -jar app.jar`. So there are defined to keep existing deployments work if they are dependent on JDK internal APIs. The attributes have been in JDK 9 for a long time and I'm not aware of any issues. If it looks like it's not your "own doing" then send an example and we'll figure it out. -Alan From paul.bakker.nl at gmail.com Thu May 11 20:25:52 2017 From: paul.bakker.nl at gmail.com (Paul Bakker) Date: Thu, 11 May 2017 13:25:52 -0700 Subject: #AddExportsInManifest In-Reply-To: References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> Message-ID: I'm a little confused by "...deployments work if they are dependent on JDK internal APIs". What does internal JDK usage have to do with opening/exporting your own packages? I would think this solves the problem that some other code (e.g. a library) requires access to application code? Also, can you elaborate why this only applies to the unnamed module and can't do the same a the --add-exports/--add-opens flags? Note that I don't actually need these features, but I want to make sure I document correctly. Paul > On May 11, 2017, at 12:10 AM, Alan Bateman wrote: > > On 11/05/2017 07:51, Paul Bakker wrote: > >> Hi Alan, >> >> What is the reason only exports/opens to unnamed are possible? >> Also, are these implemented in the current jigsaw prototype? I'm having trouble getting it to work, but that might be entirely my own doing... >> > The attributes are defined for the main application JAR when run with `java -jar app.jar`. So there are defined to keep existing deployments work if they are dependent on JDK internal APIs. > > The attributes have been in JDK 9 for a long time and I'm not aware of any issues. If it looks like it's not your "own doing" then send an example and we'll figure it out. > > -Alan From mandy.chung at oracle.com Thu May 11 20:51:00 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 11 May 2017 13:51:00 -0700 Subject: #AddExportsInManifest In-Reply-To: References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> Message-ID: <37361771-37B8-4EB1-B9A4-AA7F590968EF@oracle.com> These two attributes are defined to ease migration so that executable JARs can run with java -jar command, as is today, to avoid adding command-line options to break into encapsulation. Mandy [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AddExportsInManifest > On May 11, 2017, at 1:25 PM, Paul Bakker wrote: > > I'm a little confused by "...deployments work if they are dependent on JDK internal APIs". What does internal JDK usage have to do with opening/exporting your own packages? I would think this solves the problem that some other code (e.g. a library) requires access to application code? > Also, can you elaborate why this only applies to the unnamed module and can't do the same a the --add-exports/--add-opens flags? > > Note that I don't actually need these features, but I want to make sure I document correctly. > > Paul > > >> On May 11, 2017, at 12:10 AM, Alan Bateman wrote: >> >> On 11/05/2017 07:51, Paul Bakker wrote: >> >>> Hi Alan, >>> >>> What is the reason only exports/opens to unnamed are possible? >>> Also, are these implemented in the current jigsaw prototype? I'm having trouble getting it to work, but that might be entirely my own doing... >>> >> The attributes are defined for the main application JAR when run with `java -jar app.jar`. So there are defined to keep existing deployments work if they are dependent on JDK internal APIs. >> >> The attributes have been in JDK 9 for a long time and I'm not aware of any issues. If it looks like it's not your "own doing" then send an example and we'll figure it out. >> >> -Alan > From peter.levart at gmail.com Thu May 11 21:00:48 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 11 May 2017 23:00:48 +0200 Subject: [9] Review request: 8177566: FX user module gets IllegalAccessException from sun.reflect.misc.Trampoline In-Reply-To: <59126ABB.8090304@oracle.com> References: <5907D11D.20609@oracle.com> <1f806cdc-a241-04aa-6e4b-0b11abb1f0a9@gmail.com> <59126ABB.8090304@oracle.com> Message-ID: Hi Kevin, On 05/10/2017 03:19 AM, Kevin Rushforth wrote: > inline > > Peter Levart wrote: >> Hi Kevin, >> >> On 05/02/2017 02:21 AM, Kevin Rushforth wrote: >>> This review is being cross-posted to both openjfx-dev and jigsaw-dev. >>> >>> Please review the proposed fix for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8177566 >>> http://cr.openjdk.java.net/~kcr/8177566/webrev.00/complete-webrev/ >>> >>> Details of the fix as well as notes to reviewers are in the bug >>> report [1] (e.g., I've also generated separate webrevs for the fix >>> itself, the doc changes, and the test changes). >>> >>> -- Kevin >>> >>> [1] >>> https://bugs.openjdk.java.net/browse/JDK-8177566?focusedCommentId=14074243&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14074243 >>> >> >> I think it is very unusual to handle unqualified exports as something >> special, different from qualified exports. I know what the reasoning >> is: if a package is exported unconditionally (to everyone) then it is >> part of public API and so the trampoline may access members of that >> package on behalf of anyone. But such reasoning is just a consequence >> of the lack of a finer-grained (per-module) access support in JavaFX. >> I know it is too much to ask for JDK 9, but could JavaFX in say JDK >> 10, given current API, somehow determine on whose behalf it is making >> the trampoline access? If it could, then the trampoline could allow >> qualified exports to be effective too. > > The primary remedy is for the application to use a qualified "opens" > to the appropriate javafx module. For example, to allow the > JavaBeanXXXProperty classes the ability to access a class in your > module, your module needs to "opens my.package to javafx.base". The > only difference between what you propose and what was implemented is > qualified exports versus qualified opens, which really shouldn't be > too much of an issue for applications (such applications already need > to use qualified opens to allow access to their FXML controller class). > > The only reason we mention unconditional exports as an alternative is > for the benefit of application that happen to already have their > package exported unconditionally. > > -- Kevin I was thinking more in the direction of who the "real" accessor is when some JavaBeanXXXProperty is being used to access the bean getter/setter methods. Could it be the one invoking the JavaBeanXXXProperty.get()/.set() methods? Could it be the one invoking the JavaBeanXXXProperty.bind(ObservableValue) ? JavaBeanXXXProperty is a kind of "reflection" API with additional features. Classical Java reflection, for example, uses the "real" caller (the one invoking Method.invoke or Field.get/.set) to base access decisions on. Would this be the right approach for JavaBeanXXXProperty too? (injections with @FXML are a different story). Say for example, that module A has some Java bean classes that it would like to expose solely to module B and module B would like to bind their properties to some observables. Now module A would like to expose those bean classes to B with simple qualified exports so that no other module but B could bind or access A's bean properties. Does this make sense so far? If qualified "opens" to the appropriate javafx module is enough for JavaBeanXXXProperty to access bean properties in so opened packages, then JavaBeanXXXProperty provides a means for anyone to access those getters/setters. In my view this represents an elevation of privilege. A qualified opens to javafx then means more than just that. It means getters/setters are open to anyone who dares to use JavaBeanXXXProperty API instead of classic reflection API. What do you think? Regards, Peter From s.2016.kay at gmail.com Thu May 11 22:45:43 2017 From: s.2016.kay at gmail.com (Ess Kay) Date: Fri, 12 May 2017 08:45:43 +1000 Subject: private and non-final fields in Java 9 interfaces Message-ID: (This is not a jigsaw specific specific question but I could not find a more appropriate mailing list. The COIN list is archived. If there is a a more appropriate mailing list then please let me know.) The Java 9 compiler currently allows 1) private static and instance fields and 2) non-final static and instance fields. Is this a bug? If not then are these changes specified or mentioned anywhere? From alex.buckley at oracle.com Thu May 11 22:50:22 2017 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 11 May 2017 15:50:22 -0700 Subject: private and non-final fields in Java 9 interfaces In-Reply-To: References: Message-ID: <5914EAAE.6000003@oracle.com> compiler-dev is the right list to query javac's behavior. Support for private methods in interfaces came via JEP 213, and it sounds like you're saying private fields are allowed accidentally. Please give example source code when you write to compiler-dev. Alex On 5/11/2017 3:45 PM, Ess Kay wrote: > (This is not a jigsaw specific specific question but I could not find > a more appropriate mailing list. The COIN list is archived. If there > is a a more appropriate mailing list then please let me know.) > > The Java 9 compiler currently allows > 1) private static and instance fields and > 2) non-final static and instance fields. > Is this a bug? If not then are these changes specified or mentioned anywhere? > From david.lloyd at redhat.com Fri May 12 00:43:09 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 11 May 2017 19:43:09 -0500 Subject: Some suggested patches and improvements Message-ID: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> I've proposed five patches to the jpms-spec-experts list [1..5] for discussion. The patches are as follows: 1. Layer primitive: addExports() - mirrors the existing Module.addExports() method for ModuleLayer.Controllers 2. Layer primitive: addUses() - mirrors the existing Module.addUses() method for ModuleLayer.Controllers 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add packages to a module after it has been defined 4. Make run-time cycle checking optional 5. Add optional class loader isolation for modules on the module path Apart from #3, each of these patches is small and very low-risk, and address a clear problem. In the case of #3, I believe that the risk is very low, but am also willing to discuss modifying the patch to move the method to a "jdk.unsupported" class or in other ways. I also am maintaining a mirror of the jigsaw/jake tree as a Git repository [6] which contains these patches where they can be viewed and fiddled with on a more interactive basis. It is my belief that these small, low-cost patches will go a very long way towards mitigating many of the problems identified and shared by Red Hat, by other EG members, by other EC members, and by the community. Please have a look and let me know if you have any feedback on any of these changes. [1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000705.html [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000707.html [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000708.html [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000711.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000710.html [6] https://github.com/dmlloyd/openjdk-modules -- - DML From sstark at redhat.com Fri May 12 04:25:45 2017 From: sstark at redhat.com (Scott Stark) Date: Fri, 12 May 2017 00:25:45 -0400 (EDT) Subject: Some suggested patches and improvements In-Reply-To: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <204858181.6376666.1494563145288.JavaMail.zimbra@redhat.com> Regarding previous comments about not wanting to support such an extension of what are viewed as internal APIs, these proposed extensions could be guarded by a command line option such as --enable-jpms-experimental-api that carries warning messages along the lines of the --permit-illegal-access option, to indicate these are unsupported APIs that will be removed in the next release. That will allow other modules systems the integration they require as well as time to define public APIs that mutually agreeable. ----- Original Message ----- From: "David M. Lloyd" To: "jigsaw-dev" Sent: Thursday, May 11, 2017 5:43:09 PM Subject: Some suggested patches and improvements I've proposed five patches to the jpms-spec-experts list [1..5] for discussion. The patches are as follows: 1. Layer primitive: addExports() - mirrors the existing Module.addExports() method for ModuleLayer.Controllers 2. Layer primitive: addUses() - mirrors the existing Module.addUses() method for ModuleLayer.Controllers 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add packages to a module after it has been defined 4. Make run-time cycle checking optional 5. Add optional class loader isolation for modules on the module path Apart from #3, each of these patches is small and very low-risk, and address a clear problem. In the case of #3, I believe that the risk is very low, but am also willing to discuss modifying the patch to move the method to a "jdk.unsupported" class or in other ways. I also am maintaining a mirror of the jigsaw/jake tree as a Git repository [6] which contains these patches where they can be viewed and fiddled with on a more interactive basis. It is my belief that these small, low-cost patches will go a very long way towards mitigating many of the problems identified and shared by Red Hat, by other EG members, by other EC members, and by the community. Please have a look and let me know if you have any feedback on any of these changes. [1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000705.html [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000707.html [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000708.html [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000711.html [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000710.html [6] https://github.com/dmlloyd/openjdk-modules -- - DML From forax at univ-mlv.fr Fri May 12 06:17:35 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 12 May 2017 08:17:35 +0200 (CEST) Subject: An alternative to "restricted keywords" In-Reply-To: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> Message-ID: <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> [CC JPMS expert mailing list because, it's an important issue IMO] I've a counter proposition. I do not like your proposal because from the user point of view, '^' looks like a hack, it's not used anywhere else in the grammar. I agree that restricted keywords are not properly specified in JLS. Reading your mail, i've discovered that what i was calling restricted keywords is not what javac implements :( I agree that restricted keywords should be only enabled when parsing module-info.java I agree that doing error recovery on the way the grammar for module-info is currently implemented in javac leads to less than ideal error messages. In my opinion, both module m { requires transitive transitive; } module m { requires transitive; } should be rejected because what javac implements something more close to the javascript ASI rules than restricted keywords as currently specified by Alex. For me, a restricted keyword is a keyword which is activated if you are at a position in the grammar where it can be recognized and because it's a keyword, it tooks over an identifier. by example for module m { if the next token is 'requires', it should be recognized as a keyword because you can parse a directive 'required ...' so there is a production that will starts with the 'required' keyword. so module m { requires transitive; } should be rejected because transitive should be recognized as a keyword after requires and the compiler should report a missing module name. and module m { requires transitive transitive; } should be rejected because the grammar that parse the modifiers is defined as "a loop" so from the grammar point of view it's like module m { requires Modifier Modifier; } so the the front end of the compiler should report a missing module name and a later phase should report that there is twice the same modifier 'transitive'. I believe that with this definition of 'restricted keyword', compiler can recover error more easily and offers meaningful error message and the module-info part of the grammar is LR(1). regards, R?mi ----- Mail original ----- > De: "Stephan Herrmann" > ?: jigsaw-dev at openjdk.java.net > Envoy?: Mardi 9 Mai 2017 16:56:11 > Objet: An alternative to "restricted keywords" > (1) I understand the need for avoiding that new module-related > keywords conflict with existing code, where these words may be used > as identifiers. Moreover, it must be possible for a module declaration > to refer to packages or types thusly named. > > However, > > (2) The currently proposed "restricted keywords" are not appropriately > specified in JLS. > > (3) The currently proposed "restricted keywords" pose difficulties to > the implementation of all tools that need to parse a module declaration. > > (4) A simple alternative to "restricted keywords" exists, which has not > received the attention it deserves. > > Details: > > (2) The current specification implicitly violates the assumption that > parsing can be performed on the basis of a token stream produced by > a scanner (aka lexer). From discussion on this list we learned that > the following examples are intended to be syntactically legal: > module m { requires transitive transitive; } > module m { requires transitive; } > (Please for the moment disregard heuristic solutions, while we are > investigating whether generally "restricted keywords" is a well-defined > concept, or not.) > Of the three occurrences of "transitive", #1 is a keyword, the others > are identifiers. At the point when the parser has consumed "requires" > and now asks about classification of the word "transitive", the scanner > cannot possible answer this classification. It can only answer for sure, > after the *parser* has accepted the full declaration. Put differently, > the parser must consume more tokens than have been classified by the > Scanner. Put differently, to faithfully parse arbitrary grammars using > a concept of "restricted keywords", scanners must provide speculative > answers, which may later need to be revised by backtracking or similar > exhaustive exploration of the space of possible interpretations. > > The specification is totally silent about this fundamental change. > > > (3) "restricted keywords" pose three problems to tool implementations: > > (3.a) Any known practical approach to implement a parser with > "restricted keywords" requires to leverage heuristics, which are based > on the exact set of rules defined in the grammar. Such heuristics > reduce the look-ahead that needs to be performed by the scanner, > in order to avoid the full exhaustive exploration mentioned above. > A set of such heuristic is extremely fragile and can easily break when > later more rules are added to the grammar. This means small future > language changes can easily break any chosen strategy. > > (3.b) If parsing works for error-free input, this doesn't imply that > a parser will be able to give any useful answer for input with syntax > errors. As a worst-case example consider an arbitrary input sequence > consisting of just the two words "requires" and "transitive" in random > order and with no punctuation. > A parser will not be able to detect any structure in this sequence. > By comparison, normal keywords serve as a baseline, where parsing > typically can resume regardless of any leading garbage. > While this is not relevant for normal compilation, it is paramount > for assistive functions, which most of the time operate on incomplete > text, likely to contain even syntax errors. > Strictly speaking, any "module declaration" with syntax errors is > not a ModuleDeclaration, and thus none of the "restrictive keywords" > can be interpreted as keywords (which per JLS can only happen inside > a ModuleDeclaration). > All this means, that functionality like code completion is > systematically broken in a language using "restricted keywords". > > (3.c) Other IDE functionality assumes that small fragments of the > input text can be scanned out of context. The classical example here > is syntax highlighting but there are more examples. > Any such functionality has to be re-implemented, replacing the > highly efficient local scanning with full parsing of the input text. > For functionality that is implicitly invoked per keystroke, or on > mouse hover etc, this difference in efficiency negatively affects > the overall user experience of an IDE. > > > (4) The following proposal avoids all difficulties described above: > > * open, module, requires, transitive, exports, opens, to, uses, > provides, and with are "module words", to which the following > interpretation is applied: > * within any ordinary compilation unit, a module word is a normal > identifier. > * within a modular compilation unit, all module words are > (unconditional) keywords. > * We introduce three new auxiliary non-terminals: > LegacyPackageName: > LegacyIdentifier > LegacyPackageName . LegacyIdentifier > LegacyTypeName: > LegacyIdentifier > LegacyTypeName . LegacyIdentifier > LegacyIdentifier: > Identifier > ^open > ^module > ... > ^with > * We modify all productions in 7.7, replacing PackageName with > LegacyPackageName and replacing TypeName with LegacyTypeName. > * After parsing, each of the words '^open', '^module' etc. > is interpreted by removing the leading '^' (escape character). > > Here, '^' is chosen as the escape character following the precedent > of Xtext. Plenty of other options for this purpose are possible, too. > > > > This proposal completely satisfies the requirements (1), and avoids > all of the problems (2) and (3). There's an obvious price to pay: > users will have to add the escape character when referring to code > that uses a module word as a package name or type name. > > Not only is this a very low price compared to the benefits; one can > even argue that it also helps the human reader of a module declaration, > because it clearly marks which occurrences of a module word are indeed > identifiers. > > An IDE can easily help in interactively adding escapes where necessary. > > Finally, in this trade-off it is relevant to consider the expected > frequencies: legacy names (needing escape) will surely be the exception > - by magnitudes. So, the little price needing to be paid, will only > affect a comparatively small number of locations. > > > Stephan From christoph.langer at sap.com Fri May 12 07:16:26 2017 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 12 May 2017 07:16:26 +0000 Subject: SecurityManager.checkPackageAccess for qualified exports Message-ID: <4914956cf9844235b352aa33f1af5873@sap.com> Hi all, while playing with the security manager (using -Djava.security.manager) in Java 9 and testing platform modules that we have added specifically in our build, I came across the following thing: As we are using some stuff from jdk.internal, I get the AccessControlException: "exception access denied ("java.lang.RuntimePermission" "accessClassInPackage.jdk.internal.misc")" in several places, even if my code runs priviledged. I figured that I need to grant permission "permission java.lang.RuntimePermission "accessClassInPackage.jdk.internal.misc"" to my module. I was looking around where this restriction comes from and learned the following in the documentation of SecurityManager.checkPackageAccess: Implementation Note: This implementation also restricts all non-exported packages of modules loaded by the platform class loader or its ancestors. A "non-exported package" refers to a package that is not exported to all modules. Specifically, it refers to a package that either is not exported at all by its containing module or is exported in a qualified fashion by its containing module. Reading this, I'm wondering whether the implementation should implicitly grant package access for modules that a package in question was exported to in a qualified fashion? Now one ends up having to additionally add specific permissions which can easily be forgot. Any comments? Shouldn't that be improved? Best regards Christoph From Alan.Bateman at oracle.com Fri May 12 07:24:05 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 May 2017 08:24:05 +0100 Subject: SecurityManager.checkPackageAccess for qualified exports In-Reply-To: <4914956cf9844235b352aa33f1af5873@sap.com> References: <4914956cf9844235b352aa33f1af5873@sap.com> Message-ID: <22eed28a-0f83-54d8-d85b-615f89f5895e@oracle.com> On 12/05/2017 08:16, Langer, Christoph wrote: > Hi all, > > while playing with the security manager (using -Djava.security.manager) in Java 9 and testing platform modules that we have added specifically in our build, I came across the following thing: > > As we are using some stuff from jdk.internal, I get the AccessControlException: "exception access denied ("java.lang.RuntimePermission" "accessClassInPackage.jdk.internal.misc")" in several places, even if my code runs priviledged. I figured that I need to grant permission "permission java.lang.RuntimePermission "accessClassInPackage.jdk.internal.misc"" to my module. I was looking around where this restriction comes from and learned the following in the documentation of SecurityManager.checkPackageAccess: > > > Implementation Note: > This implementation also restricts all non-exported packages of modules loaded by the platform class loader or its ancestors. A "non-exported package" refers to a package that is not exported to all modules. Specifically, it refers to a package that either is not exported at all by its containing module or is exported in a qualified fashion by its containing module. > > Reading this, I'm wondering whether the implementation should implicitly grant package access for modules that a package in question was exported to in a qualified fashion? Now one ends up having to additionally add specific permissions which can easily be forgot. > > Any comments? Shouldn't that be improved? > I recall there was some discussion on this topic already - can you bring this to security-dev where the changes in this area were discussed/reviewed? -Alan From Alan.Bateman at oracle.com Fri May 12 07:26:07 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 May 2017 08:26:07 +0100 Subject: #AddExportsInManifest In-Reply-To: References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> Message-ID: <0d032c3a-d004-a2bc-428e-3dfef8b75193@oracle.com> On 11/05/2017 21:25, Paul Bakker wrote: > I'm a little confused by "...deployments work if they are dependent on JDK internal APIs". What does internal JDK usage have to do with opening/exporting your own packages? I would think this solves the problem that some other code (e.g. a library) requires access to application code? > Also, can you elaborate why this only applies to the unnamed module and can't do the same a the --add-exports/--add-opens flags? > Hopefully the mail that Mandy references gives the context. As an example, suppose an executable JAR that has code that is hacking into sun.awt.X11 for some reason. To keep that code working then you could do: java --add-opens java.desktop/sun.awt.X11=ALL-UNNAMED -jar app.jar but this isn't convenient for end-users that double click JAR files. The equivalent for an executable JAR is to add "Add-Opens: java.desktop/sun.awt" to its main manifest. -Alan From Alan.Bateman at oracle.com Fri May 12 08:22:54 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 May 2017 09:22:54 +0100 Subject: Some suggested patches and improvements In-Reply-To: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 12/05/2017 01:43, David M. Lloyd wrote: > I've proposed five patches to the jpms-spec-experts list [1..5] for > discussion. The patches are as follows: > > 1. Layer primitive: addExports() - mirrors the existing > Module.addExports() method for ModuleLayer.Controllers > 2. Layer primitive: addUses() - mirrors the existing Module.addUses() > method for ModuleLayer.Controllers > 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to > add packages to a module after it has been defined > 4. Make run-time cycle checking optional > 5. Add optional class loader isolation for modules on the module path #1-#3 are a subset of issue #LayerPrimitives in the JSR. I don't wish to comment on whether these make sense or not. However for #3 then you've missed several important error cases, e.g. illegal package names, or the package is already in another module defined to the class loader. There is impact in other areas too. #4 seems to be working around the outcome of issue #CyclicDependences in the JSR. I also don't wish to comment on that except to say that introducing system properties to skip specified checks is highly problematic from a conformance perspective. There is a lot more to #5, something that will become clear when you work through all the scenarios. The JSR and spec part are minor though but I'd prefer to hold off until there is more discussion on this topic in the JSR. -Alan From david.lloyd at redhat.com Fri May 12 13:31:50 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 08:31:50 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 05/12/2017 03:22 AM, Alan Bateman wrote: > On 12/05/2017 01:43, David M. Lloyd wrote: > >> I've proposed five patches to the jpms-spec-experts list [1..5] for >> discussion. The patches are as follows: >> >> 1. Layer primitive: addExports() - mirrors the existing >> Module.addExports() method for ModuleLayer.Controllers >> 2. Layer primitive: addUses() - mirrors the existing Module.addUses() >> method for ModuleLayer.Controllers >> 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to >> add packages to a module after it has been defined >> 4. Make run-time cycle checking optional >> 5. Add optional class loader isolation for modules on the module path > > #1-#3 are a subset of issue #LayerPrimitives in the JSR. I don't wish to > comment on whether these make sense or not. No problem, I'm just looking for technical feedback here. > However for #3 then you've > missed several important error cases, e.g. illegal package names, or the > package is already in another module defined to the class loader. There > is impact in other areas too. I can look into these two items; can you expand on the other impacts? > #4 seems to be working around the outcome of issue #CyclicDependences in > the JSR. I also don't wish to comment on that except to say that > introducing system properties to skip specified checks is highly > problematic from a conformance perspective. Can you explain what you mean by that? I'm more than happy to convert these into -X type arguments to the runtime (or, of course, to simply make this the standard behavior), but I thought this would be a simpler and safer approach (at least for a first pass). The only other place where I found run time cycles to be relevant is jlink and a couple of other tools, which I am in the process of developing a similar patch for. > There is a lot more to #5, something that will become clear when you > work through all the scenarios. The JSR and spec part are minor though > but I'd prefer to hold off until there is more discussion on this topic > in the JSR. I'd rather not hold off as the JSR essentially only has a couple of weeks left to live if there is not a revised PR. Could you please explain what you mean? Are you referring to jlink, jaotc, or something else? -- - DML From scolebourne at joda.org Fri May 12 14:37:38 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 12 May 2017 15:37:38 +0100 Subject: Some suggested patches and improvements In-Reply-To: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 12 May 2017 at 01:43, David M. Lloyd wrote: > 1. Layer primitive: addExports() - mirrors the existing Module.addExports() > method for ModuleLayer.Controllers > 2. Layer primitive: addUses() - mirrors the existing Module.addUses() method > for ModuleLayer.Controllers > 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add > packages to a module after it has been defined Are these a good idea? I don't know. What I do know is that the use cases for them seem to be focussed on low-level coders and framework writers, which is a very small subset of all Java developers, and a group who can work around difficulties. > 4. Make run-time cycle checking optional My opinion is that run-time cycles are inevitable. The proposed solutions (refactoring to API vs Impl) is not particularly good in an open source context. I'm also concerned that "requires static" (optional dependencies) also naturally leads to cycles But the question at this point is whether it is worth the effort to try and live without cycles for this release - after all, no-one wants cycles in their design. Since there is a workaround, via the low-level module/layer code, it feels like we should be willing to take the punt and try to live without cycles in 9 keeping it under review to add in 10. > 5. Add optional class loader isolation for modules on the module path Again, my opinion is that the isolation of modules is insufficient in Jigsaw and that hidden packages will sometimes be very messy. However, the only tool available to solve this appears to be classloaders, and they have plenty of flaws themselves. While opt-in isolation appears tempting, I'd prefer to wait and see a better solution in a later version. Maybe the answer is to deprecated class loaders and come up with something better! Given where we are, I think my preference is to see a JDK 9 release soon rather than end the JSR over these issues. Stephen From alan.bateman at oracle.com Fri May 12 15:02:02 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:02 +0000 Subject: hg: jigsaw/jake: 7 new changesets Message-ID: <201705121502.v4CF22h8029155@aojmv0008.oracle.com> Changeset: 25a364291f63 Author: lana Date: 2017-05-04 17:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/25a364291f63 Merge Changeset: 7931344eeb84 Author: ihse Date: 2017-05-05 13:56 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/7931344eeb84 8179557: Update generated Javadoc footer documentation link Reviewed-by: erikj ! make/Javadoc.gmk Changeset: a0c7c5e12113 Author: ihse Date: 2017-05-08 08:46 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/a0c7c5e12113 8179658: SetupProcessMarkdown creates long file names Reviewed-by: tbell, erikj ! make/Javadoc.gmk Changeset: 65694748f53c Author: ihse Date: 2017-05-08 14:02 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/65694748f53c 8140268: Generate link to specification license for JavaDoc API documentation Reviewed-by: erikj ! make/Javadoc.gmk Changeset: b25838a28195 Author: ihse Date: 2017-05-09 14:31 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/b25838a28195 8179879: Clarify install.sh Reviewed-by: erikj ! common/autoconf/build-aux/install.sh Changeset: a0ce1616b7f1 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/a0ce1616b7f1 Added tag jdk-9+169 for changeset b25838a28195 ! .hgtags Changeset: f1f818e27056 Author: alanb Date: 2017-05-12 14:19 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/f1f818e27056 Merge ! .hgtags ! make/Javadoc.gmk From alan.bateman at oracle.com Fri May 12 15:02:03 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:03 +0000 Subject: hg: jigsaw/jake/corba: 2 new changesets Message-ID: <201705121502.v4CF23P5029247@aojmv0008.oracle.com> Changeset: 72bb2cd3f013 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/72bb2cd3f013 Added tag jdk-9+169 for changeset b2218d41edef ! .hgtags Changeset: b88b0bf4088f Author: alanb Date: 2017-05-12 14:21 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/b88b0bf4088f Merge ! .hgtags From alan.bateman at oracle.com Fri May 12 15:02:05 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:05 +0000 Subject: hg: jigsaw/jake/nashorn: 3 new changesets Message-ID: <201705121502.v4CF25Hp029293@aojmv0008.oracle.com> Changeset: 131e25008015 Author: ihse Date: 2017-05-09 12:54 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/131e25008015 8179889: Fix typographic errors in copyright headers Reviewed-by: erikj, dholmes ! src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/codegen/SpillObjectCreator.java ! src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/codegen/TypeMap.java ! src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/SpillProperty.java ! test/script/basic/JDK-8150218.js ! test/script/basic/JDK-8170594.js ! test/script/basic/JDK-8171849.js ! test/script/basic/es6/JDK-8168373.js Changeset: 597d3dc02579 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/597d3dc02579 Added tag jdk-9+169 for changeset 131e25008015 ! .hgtags Changeset: 85cc76af3c6c Author: alanb Date: 2017-05-12 14:22 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/85cc76af3c6c Merge ! .hgtags From alan.bateman at oracle.com Fri May 12 15:02:06 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:06 +0000 Subject: hg: jigsaw/jake/jaxp: 5 new changesets Message-ID: <201705121502.v4CF26hm029339@aojmv0008.oracle.com> Changeset: 60abb1d1cd1d Author: ihse Date: 2017-05-09 12:54 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/60abb1d1cd1d 8179889: Fix typographic errors in copyright headers Reviewed-by: erikj, dholmes ! src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/xs/traversers/XSAttributeChecker.java ! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToHTMLStream.java ! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/ToXMLStream.java ! src/java.xml/share/classes/com/sun/org/apache/xml/internal/serializer/dom3/DOM3TreeWalker.java Changeset: f9abe02cceda Author: joehw Date: 2017-05-09 10:34 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/f9abe02cceda 8150256: removing xerces-related dead code Reviewed-by: lancea - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerElementHandler.java - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerFramework.java - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerSchema.java Changeset: 5d9d2a65fb26 Author: joehw Date: 2017-05-09 11:46 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/5d9d2a65fb26 8179863: Missing copyrights in some jaxp files Reviewed-by: lancea ! src/java.xml/share/classes/com/sun/org/apache/xalan/internal/xsltc/compiler/XPathParser.java ! src/java.xml/share/classes/com/sun/org/apache/xalan/internal/xsltc/compiler/sym.java Changeset: f263aa7f60c3 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/f263aa7f60c3 Added tag jdk-9+169 for changeset 5d9d2a65fb26 ! .hgtags Changeset: a3bdddfa511f Author: alanb Date: 2017-05-12 14:21 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/a3bdddfa511f Merge ! .hgtags - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerElementHandler.java - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerFramework.java - src/java.xml/share/classes/com/sun/org/apache/xerces/internal/xinclude/XPointerSchema.java From alan.bateman at oracle.com Fri May 12 15:02:07 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:07 +0000 Subject: hg: jigsaw/jake/hotspot: 5 new changesets Message-ID: <201705121502.v4CF28oa029422@aojmv0008.oracle.com> Changeset: 70548873832d Author: lana Date: 2017-05-04 17:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/70548873832d Merge Changeset: d7da8c2b8b6c Author: roland Date: 2017-04-25 09:37 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/d7da8c2b8b6c 8179070: nashorn+octane's box2d causes c2 to crash with "Bad graph detected in compute_lca_of_uses" Summary: ciTypeFlow speculates field is null but parsing uses non null constant because of concurrent class initialization Reviewed-by: thartmann ! src/share/vm/opto/parse3.cpp Changeset: 16d692be099c Author: ihse Date: 2017-05-09 12:53 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/16d692be099c 8179889: Fix typographic errors in copyright headers Reviewed-by: erikj, dholmes ! src/cpu/aarch64/vm/assembler_aarch64.cpp ! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ppc64/PPC64RegisterMap.java ! test/gc/class_unloading/TestClassUnloadingDisabled.java ! test/native/logging/test_logTagSetDescriptions.cpp ! test/native/memory/test_metachunk.cpp ! test/runtime/CommandLine/PermGenFlagsTest.java ! test/runtime/logging/ThreadLoggingTest.java ! test/runtime/logging/p2/B.jcod ! test/testlibrary/ctw/Makefile Changeset: 910e24afc502 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/910e24afc502 Added tag jdk-9+169 for changeset 16d692be099c ! .hgtags Changeset: 608adb9f402d Author: alanb Date: 2017-05-12 14:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/608adb9f402d Merge ! .hgtags ! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/ppc64/PPC64RegisterMap.java From alan.bateman at oracle.com Fri May 12 15:02:09 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:09 +0000 Subject: hg: jigsaw/jake/langtools: 5 new changesets Message-ID: <201705121502.v4CF29ci029432@aojmv0008.oracle.com> Changeset: 5daed0e904ac Author: lana Date: 2017-05-04 17:55 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/5daed0e904ac Merge Changeset: 1faee09b8da1 Author: jlahoda Date: 2017-05-09 12:22 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/1faee09b8da1 8179531: JShell: fails to provide bytecode for dynamically created lambdas Summary: Adding support for getResource(s) to the JShell's ClassLoader Reviewed-by: psandoz, rfield ! src/jdk.jshell/share/classes/jdk/jshell/execution/DefaultLoaderDelegate.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/DirectExecutionControl.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/JdiExecutionControl.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/LoaderDelegate.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/RemoteExecutionControl.java + test/jdk/jshell/GetResourceTest.java Changeset: 0e522ff8b9f5 Author: ihse Date: 2017-05-09 12:55 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/0e522ff8b9f5 8179889: Fix typographic errors in copyright headers Reviewed-by: erikj, dholmes ! src/jdk.jshell/share/classes/jdk/jshell/JShell.java ! src/jdk.jshell/share/classes/jdk/jshell/execution/JdiInitiator.java ! test/jdk/javadoc/doclet/testStylesheet/pkg/A.java ! test/jdk/javadoc/tool/treeapi/overview.html ! test/jdk/jshell/WrapperTest.java ! test/tools/javac/classreader/8171132/BadConstantValue.java ! test/tools/javac/modules/PoorChoiceForModuleNameTest.java ! test/tools/javadoc/sampleapi/res/fx.xml ! test/tools/javadoc/sampleapi/res/simple.xml ! test/tools/javadoc/sampleapi/res/tiny.xml Changeset: bc191bf0aa56 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/bc191bf0aa56 Added tag jdk-9+169 for changeset 0e522ff8b9f5 ! .hgtags Changeset: 69ae34a97515 Author: alanb Date: 2017-05-12 14:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/69ae34a97515 Merge ! .hgtags From alan.bateman at oracle.com Fri May 12 15:02:08 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:08 +0000 Subject: hg: jigsaw/jake/jaxws: 3 new changesets Message-ID: <201705121502.v4CF28iM029428@aojmv0008.oracle.com> Changeset: 912cf69806d5 Author: lancea Date: 2017-05-05 13:32 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/912cf69806d5 8179566: Add additional jaxws messages to be translated Reviewed-by: alanb, mchung + src/jdk.xml.ws/share/classes/com/sun/tools/internal/ws/resources/newmessages.properties Changeset: e75d3abe579a Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/e75d3abe579a Added tag jdk-9+169 for changeset 912cf69806d5 ! .hgtags Changeset: f5ae1e9f7870 Author: alanb Date: 2017-05-12 14:21 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/f5ae1e9f7870 Merge ! .hgtags From alan.bateman at oracle.com Fri May 12 15:02:27 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:02:27 +0000 Subject: hg: jigsaw/jake/jdk: 28 new changesets Message-ID: <201705121502.v4CF2Sj0029754@aojmv0008.oracle.com> Changeset: 88379fba79d3 Author: amlu Date: 2017-05-04 20:24 +0800 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/88379fba79d3 8023897: Replace/update/rename executeAndCatch in various tests to assertThrows Reviewed-by: dfuchs, prappo, psandoz, rriggs ! test/java/util/Arrays/ParallelPrefix.java ! test/java/util/Arrays/StreamAndSpliterator.java ! test/java/util/BitSet/stream/BitSetStreamTest.java ! test/java/util/Collections/EmptyNavigableMap.java ! test/java/util/Collections/EmptyNavigableSet.java ! test/java/util/Iterator/PrimitiveIteratorDefaults.java ! test/java/util/Map/Defaults.java ! test/java/util/Random/RandomTest.java ! test/java/util/Spliterator/SpliteratorCollisions.java ! test/java/util/Spliterator/SpliteratorFailFastTest.java ! test/java/util/Spliterator/SpliteratorTraversingAndSplittingTest.java ! test/java/util/stream/bootlib/java.base/java/util/SpliteratorTestHelper.java Changeset: 1895211a6fe9 Author: lana Date: 2017-05-04 17:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/1895211a6fe9 Merge - test/tools/launcher/modules/listmods/src/java.transaction/javax/transaction/atomic/Atomic.java Changeset: adca794fc8e2 Author: lancea Date: 2017-05-04 14:07 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/adca794fc8e2 8179634: Add JDBC 4.2 to bullet list in package.html Reviewed-by: joehw ! src/java.sql/share/classes/java/sql/package.html Changeset: 9c9b8a48cd4a Author: shshahma Date: 2017-04-27 00:08 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/9c9b8a48cd4a 8177721: Improve diagnostics in sun.management.Agent#startAgent() Reviewed-by: dfuchs, dholmes, poonam ! src/jdk.management.agent/share/classes/jdk/internal/agent/Agent.java Changeset: d6b6ea1e292e Author: poonam Date: 2017-05-04 19:25 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/d6b6ea1e292e Merge Changeset: ae8ee67505d7 Author: mchung Date: 2017-05-04 15:39 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/ae8ee67505d7 8179645: java.util.jar.Packer.newPacker and newUnpacker fails when running with security manager Reviewed-by: alanb, ksrini ! src/java.base/share/classes/com/sun/java/util/jar/pack/PropMap.java + test/java/util/jar/Pack200/SecurityTest.java Changeset: 58675e237ef1 Author: serb Date: 2017-04-24 15:15 +0300 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/58675e237ef1 8178971: Uncommon formatting and typos in java.desktop module Reviewed-by: prr, ssadetsky ! src/java.desktop/share/classes/java/awt/image/AbstractMultiResolutionImage.java ! src/java.desktop/share/classes/java/awt/package-info.java ! src/java.desktop/share/classes/javax/sound/midi/MetaMessage.java ! src/java.desktop/share/classes/javax/sound/midi/Sequence.java ! src/java.desktop/share/classes/javax/sound/midi/Sequencer.java ! src/java.desktop/share/classes/javax/sound/midi/ShortMessage.java ! src/java.desktop/share/classes/javax/sound/midi/SysexMessage.java ! src/java.desktop/share/classes/javax/sound/sampled/BooleanControl.java ! src/java.desktop/share/classes/javax/sound/sampled/FloatControl.java ! src/java.desktop/share/classes/javax/sound/sampled/LineEvent.java ! src/java.desktop/share/classes/javax/swing/JComponent.java ! src/java.desktop/share/classes/module-info.java Changeset: 601479d462cb Author: ssadetsky Date: 2017-04-24 07:10 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/601479d462cb 8178905: Undecorated frame is not painted on OEL7(Gnome3). Reviewed-by: prr, serb ! src/java.desktop/unix/classes/sun/awt/X11/XContentWindow.java Changeset: 7cb3dc0422ea Author: prr Date: 2017-04-24 10:49 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7cb3dc0422ea Merge - src/java.management/share/classes/mgmt-overview.html - src/java.security.jgss/share/classes/jgss-overview.html - src/jdk.jdi/share/classes/jdi-overview.html - src/jdk.security.auth/share/classes/jaas-overview.html - test/java/time/TEST.properties - test/java/util/BitSet/BitSetStreamTest.java Changeset: caf8ee16cea8 Author: ssadetsky Date: 2017-04-27 08:27 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/caf8ee16cea8 8179027: JComboBox too small under Windows LAF Reviewed-by: prr, alexsch ! src/java.desktop/share/classes/com/sun/java/swing/plaf/windows/WindowsComboBoxUI.java + test/javax/swing/JComboBox/WindowsComboBoxSize/WindowsComboBoxSizeTest.java Changeset: 120831a5da6f Author: ssadetsky Date: 2017-04-27 08:49 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/120831a5da6f 8140237: [TEST_BUG]Test javax/swing/plaf/nimbus/8041642/bug8041642.java fails for OEL 7 Reviewed-by: alexsch ! test/javax/swing/plaf/nimbus/8041642/bug8041642.java Changeset: 825903c4e875 Author: prr Date: 2017-04-27 14:07 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/825903c4e875 8179365: JAWT (AWT Native Interface) specification needs to be updated for JDK 9 Reviewed-by: serb, ssadetsky ! src/java.desktop/macosx/native/include/jawt_md.h ! src/java.desktop/share/classes/java/awt/package-info.java ! src/java.desktop/share/native/include/jawt.h + src/java.desktop/share/specs/AWT_Native_Interface.html Changeset: 7f76295dcd55 Author: prr Date: 2017-04-28 13:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7f76295dcd55 Merge Changeset: 3d3ee4e0f595 Author: ssadetsky Date: 2017-05-01 10:24 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/3d3ee4e0f595 8160530: [TEST-BUG] Consistent failure of java/awt/dnd/MissingEventsOnModalDialog/MissingEventsOnModalDialogTest.java Reviewed-by: yan, prr, serb ! test/java/awt/dnd/MissingEventsOnModalDialog/MissingEventsOnModalDialogTest.java Changeset: 30a722cc06ec Author: prr Date: 2017-05-02 11:24 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/30a722cc06ec Merge - test/demo/jvmti/Context.java - test/demo/jvmti/DemoRun.java - test/demo/jvmti/HeapUser.java - test/demo/jvmti/Hello.java - test/demo/jvmti/compiledMethodLoad/CompiledMethodLoadTest.java - test/demo/jvmti/gctest/BigHello.java - test/demo/jvmti/gctest/Gctest.java - test/demo/jvmti/heapTracker/HeapTrackerTest.java - test/demo/jvmti/heapViewer/HeapViewerTest.java - test/demo/jvmti/minst/MinstExample.java - test/demo/jvmti/minst/MinstTest.java - test/demo/jvmti/versionCheck/FailsWhenJvmtiVersionDiffers.java - test/demo/jvmti/waiters/WaitersTest.java Changeset: 556f1721939f Author: ssadetsky Date: 2017-05-02 12:10 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/556f1721939f 8159902: OGL surfaces are not HiDPI compatible on Linux/Solaris Reviewed-by: prr, serb ! src/java.desktop/unix/classes/sun/java2d/opengl/GLXSurfaceData.java ! test/javax/swing/JInternalFrame/8069348/bug8069348.java Changeset: e07effddeebf Author: ssadetsky Date: 2017-05-02 13:26 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e07effddeebf 8178984: Unnecessary angle brackets in the Line2D::intersectsLine() javadoc. Reviewed-by: prr, jdv ! src/java.desktop/share/classes/java/awt/geom/Line2D.java Changeset: 4ad40491266b Author: prr Date: 2017-05-05 08:34 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/4ad40491266b Merge - test/tools/launcher/modules/listmods/src/java.transaction/javax/transaction/atomic/Atomic.java Changeset: afac00be2a93 Author: shurailine Date: 2017-05-05 14:59 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/afac00be2a93 8179852: Remove references to demo tests from TEST.groups Reviewed-by: bpb ! test/TEST.groups Changeset: 207f458f39cb Author: serb Date: 2017-05-06 14:58 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/207f458f39cb 8179596: Update java.desktop to be HTML-5 friendly Reviewed-by: prr, amenkov ! src/java.desktop/share/classes/java/awt/AlphaComposite.java ! src/java.desktop/share/classes/java/awt/Graphics2D.java ! src/java.desktop/share/classes/java/awt/GridBagLayout.java ! src/java.desktop/share/classes/java/awt/GridLayout.java ! src/java.desktop/share/classes/java/awt/LinearGradientPaint.java ! src/java.desktop/share/classes/java/awt/MenuBar.java ! src/java.desktop/share/classes/java/awt/RadialGradientPaint.java ! src/java.desktop/share/classes/java/awt/Rectangle.java ! src/java.desktop/share/classes/java/awt/Shape.java ! src/java.desktop/share/classes/java/awt/dnd/DragSourceContext.java ! src/java.desktop/share/classes/java/awt/font/TextAttribute.java ! src/java.desktop/share/classes/java/awt/geom/AffineTransform.java ! src/java.desktop/share/classes/java/awt/geom/Arc2D.java ! src/java.desktop/share/classes/java/awt/geom/Path2D.java ! src/java.desktop/share/classes/java/awt/im/spi/package-info.java ! src/java.desktop/share/classes/java/awt/image/DataBufferByte.java ! src/java.desktop/share/classes/java/awt/image/DataBufferDouble.java ! src/java.desktop/share/classes/java/awt/image/DataBufferFloat.java ! src/java.desktop/share/classes/java/awt/image/DataBufferInt.java ! src/java.desktop/share/classes/java/awt/image/DataBufferShort.java ! src/java.desktop/share/classes/java/awt/image/DataBufferUShort.java ! src/java.desktop/share/classes/java/awt/image/IndexColorModel.java ! src/java.desktop/share/classes/javax/accessibility/package-info.java ! src/java.desktop/share/classes/javax/imageio/plugins/tiff/TIFFDirectory.java ! src/java.desktop/share/classes/javax/imageio/plugins/tiff/TIFFField.java ! src/java.desktop/share/classes/javax/print/MimeType.java ! src/java.desktop/share/classes/javax/print/attribute/standard/Finishings.java ! src/java.desktop/share/classes/javax/print/attribute/standard/MultipleDocumentHandling.java ! src/java.desktop/share/classes/javax/sound/midi/MidiMessage.java ! src/java.desktop/share/classes/javax/sound/midi/VoiceStatus.java ! src/java.desktop/share/classes/javax/sound/sampled/FloatControl.java ! src/java.desktop/share/classes/javax/sound/sampled/SourceDataLine.java ! src/java.desktop/share/classes/javax/sound/sampled/TargetDataLine.java ! src/java.desktop/share/classes/javax/swing/Action.java ! src/java.desktop/share/classes/javax/swing/BoxLayout.java ! src/java.desktop/share/classes/javax/swing/DefaultListCellRenderer.java ! src/java.desktop/share/classes/javax/swing/JLayeredPane.java ! src/java.desktop/share/classes/javax/swing/JList.java ! src/java.desktop/share/classes/javax/swing/JOptionPane.java ! src/java.desktop/share/classes/javax/swing/JRootPane.java ! src/java.desktop/share/classes/javax/swing/JScrollPane.java ! src/java.desktop/share/classes/javax/swing/JTree.java ! src/java.desktop/share/classes/javax/swing/LookAndFeel.java ! src/java.desktop/share/classes/javax/swing/SizeSequence.java ! src/java.desktop/share/classes/javax/swing/plaf/basic/BasicTextUI.java ! src/java.desktop/share/classes/javax/swing/plaf/metal/DefaultMetalTheme.java ! src/java.desktop/share/classes/javax/swing/plaf/metal/MetalLookAndFeel.java ! src/java.desktop/share/classes/javax/swing/table/DefaultTableCellRenderer.java ! src/java.desktop/share/classes/javax/swing/text/html/HTMLDocument.java ! src/java.desktop/share/classes/javax/swing/text/html/HTMLEditorKit.java ! src/java.desktop/share/classes/javax/swing/tree/DefaultTreeCellRenderer.java ! src/java.desktop/share/classes/javax/swing/undo/UndoManager.java Changeset: ca5d05dc27dd Author: xiaofeya Date: 2017-05-07 19:01 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/ca5d05dc27dd 8178912: Remove sample/chatserver/ChatTest.java and sample/mergesort/MergeSortTest.java Reviewed-by: psandoz ! test/ProblemList.txt ! test/TEST.groups - test/sample/TEST.properties - test/sample/chatserver/ChatTest.java - test/sample/mergesort/MergeSortTest.java Changeset: e81bd83cd22a Author: mchung Date: 2017-05-08 21:21 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e81bd83cd22a 8020801: Apply the restriction of invoking MethodHandles.lookup to j.l.r.Method.invoke Reviewed-by: plevart, psandoz ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java + test/java/lang/invoke/lookup/ReflectiveLookupTest.java + test/java/lang/invoke/lookup/java.base/java/lang/LookupTest.java Changeset: d17577d4839b Author: ihse Date: 2017-05-09 12:57 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/d17577d4839b 8179889: Fix typographic errors in copyright headers Reviewed-by: erikj, dholmes ! src/java.base/macosx/native/libnet/DefaultProxySelector.c ! src/java.base/share/classes/com/sun/crypto/provider/GCTR.java ! src/java.base/share/classes/sun/security/util/ConstraintsParameters.java ! src/java.base/share/native/libjimage/endian.hpp ! src/java.base/share/native/libjimage/imageDecompressor.cpp ! src/java.base/share/native/libjimage/imageDecompressor.hpp ! src/java.base/share/native/libjimage/imageFile.hpp ! src/java.base/share/native/libjimage/inttypes.hpp ! src/java.base/share/native/libjimage/jimage.hpp ! src/java.base/share/native/libjimage/osSupport.hpp ! src/java.desktop/share/classes/sun/swing/LightweightContent.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/WindowUpdateSender.java ! test/com/apple/laf/ScreenMenu/ScreenMenuMemoryLeakTest.java ! test/com/sun/crypto/provider/Cipher/AES/TestCICOWithGCMAndAAD.java ! test/com/sun/jdi/ArrayLengthDumpTest.sh ! test/com/sun/jdi/BreakpointWithFullGC.sh ! test/com/sun/jdi/CatchAllTest.sh ! test/com/sun/jdi/CatchCaughtTest.sh ! test/com/sun/jdi/CatchPatternTest.sh ! test/com/sun/jdi/CommandCommentDelimiter.sh ! test/com/sun/jdi/DeferredStepTest.sh ! test/com/sun/jdi/DeoptimizeWalk.sh ! test/com/sun/jdi/EvalArgs.sh ! test/com/sun/jdi/GetLocalVariables3Test.sh ! test/com/sun/jdi/GetLocalVariables4Test.sh ! test/com/sun/jdi/JdbExprTest.sh ! test/com/sun/jdi/JdbLockTest.sh ! test/com/sun/jdi/MixedSuspendTest.sh ! test/com/sun/jdi/NullLocalVariable.sh ! test/com/sun/jdi/NullThreadGroupNameTest.java ! test/com/sun/jdi/RedefineChangeClassOrder.sh ! test/com/sun/jdi/RedefineException.sh ! test/com/sun/jdi/RedefineImplementor.sh ! test/com/sun/jdi/RedefineIntConstantToLong.sh ! test/com/sun/management/GarbageCollectorMXBean/LastGCInfo.java ! test/com/sun/net/httpserver/SimpleHttpServerTest.java ! test/com/sun/net/httpserver/StopNoStartTest.java ! test/com/sun/net/httpserver/bugs/B6433018.java ! test/com/sun/tools/attach/PermissionTest.java ! test/com/sun/tools/attach/ProviderTest.java ! test/com/sun/tools/attach/StartManagementAgent.java ! test/com/sun/tools/attach/TempDirTest.java ! test/java/awt/Choice/UnfocusableToplevel/UnfocusableToplevel.java ! test/java/awt/Clipboard/HTMLTransferTest/HTMLTransferTest.html ! test/java/awt/Clipboard/HTMLTransferTest/HTMLTransferTest.java ! test/java/awt/Dialog/NestedDialogs/Modal/NestedModalDialogTest.java ! test/java/awt/Dialog/NestedDialogs/Modeless/NestedModelessDialogTest.java ! test/java/awt/EventQueue/MainAppContext/MainAppContext.java ! test/java/awt/FileDialog/RegexpFilterTest/RegexpFilterTest.html ! test/java/awt/Focus/WindowIsFocusableAccessByThreadsTest/WindowIsFocusableAccessByThreadsTest.java ! test/java/awt/Frame/FrameResize/ShowChildWhileResizingTest.java ! test/java/awt/FullScreen/AltTabCrashTest/AltTabCrashTest.java ! test/java/awt/LightweightComponent/LightweightEventTest/LightweightEventTest.java ! test/java/awt/Mixing/AWT_Mixing/GlassPaneOverlappingTestBase.java ! test/java/awt/Mixing/AWT_Mixing/SimpleOverlappingTestBase.java ! test/java/awt/Mouse/MaximizedFrameTest/MaximizedFrameTest.java ! test/java/awt/MouseInfo/JContainerMousePositionTest.java ! test/java/awt/Toolkit/AutoShutdown/EventQueuePush/EventQueuePushAutoshutdown.sh ! test/java/awt/Toolkit/ToolkitPropertyTest/bug7129133.java ! test/java/awt/Window/FindOwner/FindOwnerTest.java ! test/java/awt/dnd/DisposeFrameOnDragCrash/DisposeFrameOnDragTest.java ! test/java/awt/event/MouseEvent/DisabledComponents/DisabledComponentsTest.java ! test/java/awt/font/TextLayout/ArabicDiacriticTest.java ! test/java/awt/font/TextLayout/MissingCodePointLayoutTest.java ! test/java/io/RandomAccessFile/FileLengthTest.java ! test/java/lang/instrument/RedefineBigClass.sh ! test/java/lang/invoke/VarHandles/X-VarHandleTestMethodType.java.template ! test/java/net/httpclient/ProxyAuthTest.java ! test/java/nio/file/WatchService/DeleteInterference.java ! test/java/nio/file/attribute/BasicFileAttributeView/UnixSocketFile.java ! test/java/security/AccessController/LimitedDoPrivilegedWithNullPerms.java ! test/java/security/AccessController/LimitedDoPrivilegedWithThread.java ! test/java/time/test/java/time/temporal/TestIsoWeekFields.java ! test/java/util/ResourceBundle/getBaseBundleName/resources/PropertyBundle.properties ! test/java/util/Spliterator/SpliteratorLateBindingFailFastHelper.java ! test/java/util/logging/Logger/logrb/resources/PropertyBundle.properties ! test/java/util/logging/Logger/setResourceBundle/resources/PropertyBundle.properties ! test/javax/management/loading/LibraryLoader/LibraryLoaderTest.java ! test/javax/management/monitor/ThreadPoolAccTest.java ! test/javax/net/ssl/TLS/CipherTestUtils.java ! test/javax/net/ssl/TLS/JSSEClient.java ! test/javax/net/ssl/TLS/JSSEServer.java ! test/javax/net/ssl/TLS/TestJSSE.java ! test/javax/net/ssl/TLS/TestJSSEClientDefaultProtocol.java ! test/javax/net/ssl/TLS/TestJSSEClientProtocol.java ! test/javax/net/ssl/TLS/TestJSSENoCommonProtocols.java ! test/javax/net/ssl/TLS/TestJSSEServerProtocol.java ! test/javax/print/attribute/TestUnsupportedResolution.java ! test/javax/security/auth/Subject/Generic.java ! test/javax/security/auth/login/JAASConfigSyntaxCheck/JAASConfigSyntaxTest.java ! test/javax/security/auth/login/JAASConfigSyntaxCheck/SampleLoginModule.java ! test/javax/swing/JComboBox/6632953/bug6632953.java ! test/javax/swing/JFileChooser/4150029/bug4150029.html ! test/javax/swing/JInternalFrame/Test6325652.java ! test/javax/swing/JMenu/8072900/WrongSelectionOnMouseOver.java ! test/javax/swing/JPopupMenu/6694823/bug6694823.java ! test/javax/swing/plaf/synth/SynthButtonUI/6276188/bug6276188.java ! test/javax/swing/text/html/parser/Parser/8028616/bug8028616.java ! test/sun/awt/dnd/8024061/bug8024061.java ! test/sun/net/idn/NFS4StringPrep.java ! test/sun/security/krb5/auto/BogusKDC.java ! test/sun/security/krb5/auto/NoAddresses.java ! test/sun/security/krb5/auto/Renew.java ! test/sun/security/krb5/auto/Renewal.java ! test/sun/security/krb5/auto/SSLwithPerms.java ! test/sun/security/krb5/canonicalize/Test.java ! test/sun/security/mscapi/CastError.java ! test/sun/security/mscapi/PrngSlow.java ! test/sun/security/mscapi/PublicKeyInterop.sh ! test/sun/security/pkcs12/StoreSecretKeyTest.java ! test/sun/security/provider/SeedGenerator/SeedGeneratorChoice.java ! test/sun/security/tools/jarsigner/nameclash.sh ! test/sun/security/x509/URICertStore/SocksProxy.java ! test/tools/pack200/MultiRelease.java Changeset: b8207b069bc2 Author: ihse Date: 2017-05-09 13:19 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/b8207b069bc2 8179692: Move RMI spec to specs directory Reviewed-by: erikj ! src/java.rmi/share/classes/java/rmi/Remote.java ! src/java.rmi/share/classes/java/rmi/server/UnicastRemoteObject.java Changeset: e96547bd31cc Author: bpb Date: 2017-05-09 07:40 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e96547bd31cc 8179662: OutputStreamWriter javadocs states that you can set the buffer size but there is no way to do that Summary: Change specification verbiage to match longstanding behavior Reviewed-by: briangoetz, chegar, lancea, rriggs ! src/java.base/share/classes/java/io/OutputStreamWriter.java Changeset: 177436a54ca1 Author: mchung Date: 2017-05-09 10:34 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/177436a54ca1 8179950: Custom system class loader using Enum.valueOf in its initialization triggers java.lang.InternalError Reviewed-by: alanb, dfuchs, lancea ! src/java.base/share/classes/jdk/internal/reflect/Reflection.java + test/java/lang/ClassLoader/CustomSystemLoader/CustomLoader.java + test/java/lang/ClassLoader/CustomSystemLoader/InitSystemLoaderTest.java Changeset: 30fc0ab51f39 Author: lana Date: 2017-05-11 16:26 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/30fc0ab51f39 Added tag jdk-9+169 for changeset 177436a54ca1 ! .hgtags Changeset: efff9bc68313 Author: alanb Date: 2017-05-12 14:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/efff9bc68313 Merge ! .hgtags ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/jdk/internal/reflect/Reflection.java ! src/java.base/share/native/libjimage/endian.hpp ! src/java.base/share/native/libjimage/imageDecompressor.cpp ! src/java.base/share/native/libjimage/imageDecompressor.hpp ! src/java.base/share/native/libjimage/imageFile.hpp ! src/java.base/share/native/libjimage/jimage.hpp ! src/java.base/share/native/libjimage/osSupport.hpp ! src/java.desktop/share/classes/module-info.java ! src/jdk.management.agent/share/classes/jdk/internal/agent/Agent.java ! test/ProblemList.txt ! test/TEST.groups ! test/com/sun/management/GarbageCollectorMXBean/LastGCInfo.java ! test/com/sun/tools/attach/PermissionTest.java ! test/com/sun/tools/attach/ProviderTest.java ! test/com/sun/tools/attach/StartManagementAgent.java ! test/com/sun/tools/attach/TempDirTest.java ! test/javax/net/ssl/TLS/TestJSSE.java - test/sample/TEST.properties - test/sample/chatserver/ChatTest.java - test/sample/mergesort/MergeSortTest.java ! test/sun/security/krb5/auto/Renew.java ! test/sun/security/krb5/auto/Renewal.java ! test/sun/security/mscapi/PublicKeyInterop.sh From scolebourne at joda.org Fri May 12 15:23:08 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 12 May 2017 16:23:08 +0100 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <20170505074155.424443679@eggemoggin.niobe.net> References: <20170504103906.689386446@eggemoggin.niobe.net> <20170505074155.424443679@eggemoggin.niobe.net> Message-ID: On 5 May 2017 at 15:41, wrote: > I suspect what you really mean is "Never publish JARs that refer to > automatic modules that do not have `Automatic-Module-Name` attributes". > In general that's good advice. It might even be reasonable for managers > of artifact repositories to insist upon it, although I now understand > that that could be problematic for a repository as popular as Maven > Central. I think this is the tricky part. Lets review where we are. Firstly, we now have super-package reverse-DNS names and MANIFEST entries. Both of these are a very good thing. Secondly, a module with a module-info can now depend on three things: - an explicit module with a module-info (good) - an automatic module with MANIFEST entry (good) - an automatic module with name implied from filename (not good) Thirdly, given the nature of Maven Central, it is not realistic to prevent modules with dependencies on names implied from filenames getting into Central. My remaining concerns with automatic modules are primarily about this last point - we have to accept that there will be modules with "bad dependencies" in Maven Central. For example, the graph will end up with a dependency on "guava" that breaks when Guava is properly modularized and uses "com.google.common" as the module name. The result is a graph that does not resolve. While I and others will try and communicate this issue to the community, I would prefer that the issue simply could not occur. However, it has been pointed out to me off-list that there is a way to workaround the name change that I had not fully appreciated before. Given this broken module graph state: module com.foo.application { requires guava; } module com.google.common { } it is possible to fix it by introducing an additional module as follows: module guava { requires transient com.google.common; } With this additional "renaming shim module" in the graph, everything now works again. Given that there is a potential workaround, I am not as concerned as I previously was about this issue. Ideally, the creation of this "renaming shim module" should be part of the JDK (effectively it is a kind of alias, and the JDK is best placed to manage this). However, it should also be possible for tools like Maven to provide the shim dynamically when needed. It would even be possible to release such a shim module to Maven Central and depend on it in the usual way. While this is still a workaround to an issue I wish didn't exist in the first place, it is a viable solution that I think allows automatic modules to proceed as is (given the JSR/release timeline pressure). Stephen From alan.bateman at oracle.com Fri May 12 15:42:35 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Fri, 12 May 2017 15:42:35 +0000 Subject: hg: jigsaw/jake/jdk: 3 new changesets Message-ID: <201705121542.v4CFgaAe027372@aojmv0008.oracle.com> Changeset: c59d273ab6db Author: alanb Date: 2017-05-12 14:37 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/c59d273ab6db Fix javadoc links ! src/java.base/share/classes/java/lang/ModuleLayer.java ! src/java.base/share/classes/jdk/internal/loader/Loader.java Changeset: 5597f61c7b5a Author: alanb Date: 2017-05-12 16:24 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/5597f61c7b5a More SL cleanup and exception handling ! src/java.base/share/classes/java/util/ServiceLoader.java Changeset: a6c4add6c917 Author: alanb Date: 2017-05-12 16:26 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/a6c4add6c917 Merge - test/sample/TEST.properties - test/sample/chatserver/ChatTest.java - test/sample/mergesort/MergeSortTest.java From peter.levart at gmail.com Fri May 12 16:08:58 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 12 May 2017 18:08:58 +0200 Subject: An alternative to "restricted keywords" In-Reply-To: <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> Message-ID: <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> Hi Remi, On 05/12/2017 08:17 AM, Remi Forax wrote: > [CC JPMS expert mailing list because, it's an important issue IMO] > > I've a counter proposition. > > I do not like your proposal because from the user point of view, '^' looks like a hack, it's not used anywhere else in the grammar. > I agree that restricted keywords are not properly specified in JLS. Reading your mail, i've discovered that what i was calling restricted keywords is not what javac implements :( > I agree that restricted keywords should be only enabled when parsing module-info.java > I agree that doing error recovery on the way the grammar for module-info is currently implemented in javac leads to less than ideal error messages. > > In my opinion, both > module m { requires transitive transitive; } > module m { requires transitive; } > should be rejected because what javac implements something more close to the javascript ASI rules than restricted keywords as currently specified by Alex. > > For me, a restricted keyword is a keyword which is activated if you are at a position in the grammar where it can be recognized and because it's a keyword, it tooks over an identifier. > by example for > module m { > if the next token is 'requires', it should be recognized as a keyword because you can parse a directive 'required ...' so there is a production that will starts with the 'required' keyword. > > so > module m { requires transitive; } > should be rejected because transitive should be recognized as a keyword after requires and the compiler should report a missing module name. > > and > module m { requires transitive transitive; } > should be rejected because the grammar that parse the modifiers is defined as "a loop" so from the grammar point of view it's like > module m { requires Modifier Modifier; } > so the the front end of the compiler should report a missing module name and a later phase should report that there is twice the same modifier 'transitive'. > > I believe that with this definition of 'restricted keyword', compiler can recover error more easily and offers meaningful error message and the module-info part of the grammar is LR(1). This will make "requires", "uses", "provides", "with", "to", "static", "transitive", "exports", etc .... all illegal module names. Ok, no big deal, because there are no module names yet (apart from JDK modules and those are named differently). But... What about: module m { exports transitive; } Here 'transitive' is an existing package name for example. Who guarantees that there are no packages out there with names matching restricted keywords? Current restriction for modules is that they can not have an unnamed package. Do we want to restrict package names a module can export too? Stephan's solution does not have this problem. Regards, Peter > > regards, > R?mi > > ----- Mail original ----- >> De: "Stephan Herrmann" >> ?: jigsaw-dev at openjdk.java.net >> Envoy?: Mardi 9 Mai 2017 16:56:11 >> Objet: An alternative to "restricted keywords" >> (1) I understand the need for avoiding that new module-related >> keywords conflict with existing code, where these words may be used >> as identifiers. Moreover, it must be possible for a module declaration >> to refer to packages or types thusly named. >> >> However, >> >> (2) The currently proposed "restricted keywords" are not appropriately >> specified in JLS. >> >> (3) The currently proposed "restricted keywords" pose difficulties to >> the implementation of all tools that need to parse a module declaration. >> >> (4) A simple alternative to "restricted keywords" exists, which has not >> received the attention it deserves. >> >> Details: >> >> (2) The current specification implicitly violates the assumption that >> parsing can be performed on the basis of a token stream produced by >> a scanner (aka lexer). From discussion on this list we learned that >> the following examples are intended to be syntactically legal: >> module m { requires transitive transitive; } >> module m { requires transitive; } >> (Please for the moment disregard heuristic solutions, while we are >> investigating whether generally "restricted keywords" is a well-defined >> concept, or not.) >> Of the three occurrences of "transitive", #1 is a keyword, the others >> are identifiers. At the point when the parser has consumed "requires" >> and now asks about classification of the word "transitive", the scanner >> cannot possible answer this classification. It can only answer for sure, >> after the *parser* has accepted the full declaration. Put differently, >> the parser must consume more tokens than have been classified by the >> Scanner. Put differently, to faithfully parse arbitrary grammars using >> a concept of "restricted keywords", scanners must provide speculative >> answers, which may later need to be revised by backtracking or similar >> exhaustive exploration of the space of possible interpretations. >> >> The specification is totally silent about this fundamental change. >> >> >> (3) "restricted keywords" pose three problems to tool implementations: >> >> (3.a) Any known practical approach to implement a parser with >> "restricted keywords" requires to leverage heuristics, which are based >> on the exact set of rules defined in the grammar. Such heuristics >> reduce the look-ahead that needs to be performed by the scanner, >> in order to avoid the full exhaustive exploration mentioned above. >> A set of such heuristic is extremely fragile and can easily break when >> later more rules are added to the grammar. This means small future >> language changes can easily break any chosen strategy. >> >> (3.b) If parsing works for error-free input, this doesn't imply that >> a parser will be able to give any useful answer for input with syntax >> errors. As a worst-case example consider an arbitrary input sequence >> consisting of just the two words "requires" and "transitive" in random >> order and with no punctuation. >> A parser will not be able to detect any structure in this sequence. >> By comparison, normal keywords serve as a baseline, where parsing >> typically can resume regardless of any leading garbage. >> While this is not relevant for normal compilation, it is paramount >> for assistive functions, which most of the time operate on incomplete >> text, likely to contain even syntax errors. >> Strictly speaking, any "module declaration" with syntax errors is >> not a ModuleDeclaration, and thus none of the "restrictive keywords" >> can be interpreted as keywords (which per JLS can only happen inside >> a ModuleDeclaration). >> All this means, that functionality like code completion is >> systematically broken in a language using "restricted keywords". >> >> (3.c) Other IDE functionality assumes that small fragments of the >> input text can be scanned out of context. The classical example here >> is syntax highlighting but there are more examples. >> Any such functionality has to be re-implemented, replacing the >> highly efficient local scanning with full parsing of the input text. >> For functionality that is implicitly invoked per keystroke, or on >> mouse hover etc, this difference in efficiency negatively affects >> the overall user experience of an IDE. >> >> >> (4) The following proposal avoids all difficulties described above: >> >> * open, module, requires, transitive, exports, opens, to, uses, >> provides, and with are "module words", to which the following >> interpretation is applied: >> * within any ordinary compilation unit, a module word is a normal >> identifier. >> * within a modular compilation unit, all module words are >> (unconditional) keywords. >> * We introduce three new auxiliary non-terminals: >> LegacyPackageName: >> LegacyIdentifier >> LegacyPackageName . LegacyIdentifier >> LegacyTypeName: >> LegacyIdentifier >> LegacyTypeName . LegacyIdentifier >> LegacyIdentifier: >> Identifier >> ^open >> ^module >> ... >> ^with >> * We modify all productions in 7.7, replacing PackageName with >> LegacyPackageName and replacing TypeName with LegacyTypeName. >> * After parsing, each of the words '^open', '^module' etc. >> is interpreted by removing the leading '^' (escape character). >> >> Here, '^' is chosen as the escape character following the precedent >> of Xtext. Plenty of other options for this purpose are possible, too. >> >> >> >> This proposal completely satisfies the requirements (1), and avoids >> all of the problems (2) and (3). There's an obvious price to pay: >> users will have to add the escape character when referring to code >> that uses a module word as a package name or type name. >> >> Not only is this a very low price compared to the benefits; one can >> even argue that it also helps the human reader of a module declaration, >> because it clearly marks which occurrences of a module word are indeed >> identifiers. >> >> An IDE can easily help in interactively adding escapes where necessary. >> >> Finally, in this trade-off it is relevant to consider the expected >> frequencies: legacy names (needing escape) will surely be the exception >> - by magnitudes. So, the little price needing to be paid, will only >> affect a comparatively small number of locations. >> >> >> Stephan From greggwon at cox.net Fri May 12 16:15:44 2017 From: greggwon at cox.net (Gregg Wonderly) Date: Fri, 12 May 2017 11:15:44 -0500 Subject: An alternative to "restricted keywords" In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> Message-ID: Or, would it make sense to make the module name require quotes around it? The subtlety of this notation looking JSON like, and yet being something new, makes me wonder if it should not just be a JSON based structure module : [ { m : { requires : { transitive : [ ?transitive? ] } } }, { n : { requires : { transitive : [ ?m? ] } } } ] It would certainly provide a huge change in tooling possibilities. But, it would add some additional requirements for a JSON parser. Gregg > On May 12, 2017, at 11:08 AM, Peter Levart wrote: > > Hi Remi, > > On 05/12/2017 08:17 AM, Remi Forax wrote: >> [CC JPMS expert mailing list because, it's an important issue IMO] >> >> I've a counter proposition. >> >> I do not like your proposal because from the user point of view, '^' looks like a hack, it's not used anywhere else in the grammar. >> I agree that restricted keywords are not properly specified in JLS. Reading your mail, i've discovered that what i was calling restricted keywords is not what javac implements :( >> I agree that restricted keywords should be only enabled when parsing module-info.java >> I agree that doing error recovery on the way the grammar for module-info is currently implemented in javac leads to less than ideal error messages. >> >> In my opinion, both >> module m { requires transitive transitive; } >> module m { requires transitive; } >> should be rejected because what javac implements something more close to the javascript ASI rules than restricted keywords as currently specified by Alex. >> >> For me, a restricted keyword is a keyword which is activated if you are at a position in the grammar where it can be recognized and because it's a keyword, it tooks over an identifier. >> by example for >> module m { >> if the next token is 'requires', it should be recognized as a keyword because you can parse a directive 'required ...' so there is a production that will starts with the 'required' keyword. >> >> so >> module m { requires transitive; } >> should be rejected because transitive should be recognized as a keyword after requires and the compiler should report a missing module name. >> and >> module m { requires transitive transitive; } >> should be rejected because the grammar that parse the modifiers is defined as "a loop" so from the grammar point of view it's like >> module m { requires Modifier Modifier; } >> so the the front end of the compiler should report a missing module name and a later phase should report that there is twice the same modifier 'transitive'. >> >> I believe that with this definition of 'restricted keyword', compiler can recover error more easily and offers meaningful error message and the module-info part of the grammar is LR(1). > > This will make "requires", "uses", "provides", "with", "to", "static", "transitive", "exports", etc .... all illegal module names. Ok, no big deal, because there are no module names yet (apart from JDK modules and those are named differently). But... > > What about: > > module m { exports transitive; } > > Here 'transitive' is an existing package name for example. Who guarantees that there are no packages out there with names matching restricted keywords? Current restriction for modules is that they can not have an unnamed package. Do we want to restrict package names a module can export too? > > Stephan's solution does not have this problem. > > Regards, Peter > >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "Stephan Herrmann" >>> ?: jigsaw-dev at openjdk.java.net >>> Envoy?: Mardi 9 Mai 2017 16:56:11 >>> Objet: An alternative to "restricted keywords" >>> (1) I understand the need for avoiding that new module-related >>> keywords conflict with existing code, where these words may be used >>> as identifiers. Moreover, it must be possible for a module declaration >>> to refer to packages or types thusly named. >>> >>> However, >>> >>> (2) The currently proposed "restricted keywords" are not appropriately >>> specified in JLS. >>> >>> (3) The currently proposed "restricted keywords" pose difficulties to >>> the implementation of all tools that need to parse a module declaration. >>> >>> (4) A simple alternative to "restricted keywords" exists, which has not >>> received the attention it deserves. >>> >>> Details: >>> >>> (2) The current specification implicitly violates the assumption that >>> parsing can be performed on the basis of a token stream produced by >>> a scanner (aka lexer). From discussion on this list we learned that >>> the following examples are intended to be syntactically legal: >>> module m { requires transitive transitive; } >>> module m { requires transitive; } >>> (Please for the moment disregard heuristic solutions, while we are >>> investigating whether generally "restricted keywords" is a well-defined >>> concept, or not.) >>> Of the three occurrences of "transitive", #1 is a keyword, the others >>> are identifiers. At the point when the parser has consumed "requires" >>> and now asks about classification of the word "transitive", the scanner >>> cannot possible answer this classification. It can only answer for sure, >>> after the *parser* has accepted the full declaration. Put differently, >>> the parser must consume more tokens than have been classified by the >>> Scanner. Put differently, to faithfully parse arbitrary grammars using >>> a concept of "restricted keywords", scanners must provide speculative >>> answers, which may later need to be revised by backtracking or similar >>> exhaustive exploration of the space of possible interpretations. >>> >>> The specification is totally silent about this fundamental change. >>> >>> >>> (3) "restricted keywords" pose three problems to tool implementations: >>> >>> (3.a) Any known practical approach to implement a parser with >>> "restricted keywords" requires to leverage heuristics, which are based >>> on the exact set of rules defined in the grammar. Such heuristics >>> reduce the look-ahead that needs to be performed by the scanner, >>> in order to avoid the full exhaustive exploration mentioned above. >>> A set of such heuristic is extremely fragile and can easily break when >>> later more rules are added to the grammar. This means small future >>> language changes can easily break any chosen strategy. >>> >>> (3.b) If parsing works for error-free input, this doesn't imply that >>> a parser will be able to give any useful answer for input with syntax >>> errors. As a worst-case example consider an arbitrary input sequence >>> consisting of just the two words "requires" and "transitive" in random >>> order and with no punctuation. >>> A parser will not be able to detect any structure in this sequence. >>> By comparison, normal keywords serve as a baseline, where parsing >>> typically can resume regardless of any leading garbage. >>> While this is not relevant for normal compilation, it is paramount >>> for assistive functions, which most of the time operate on incomplete >>> text, likely to contain even syntax errors. >>> Strictly speaking, any "module declaration" with syntax errors is >>> not a ModuleDeclaration, and thus none of the "restrictive keywords" >>> can be interpreted as keywords (which per JLS can only happen inside >>> a ModuleDeclaration). >>> All this means, that functionality like code completion is >>> systematically broken in a language using "restricted keywords". >>> >>> (3.c) Other IDE functionality assumes that small fragments of the >>> input text can be scanned out of context. The classical example here >>> is syntax highlighting but there are more examples. >>> Any such functionality has to be re-implemented, replacing the >>> highly efficient local scanning with full parsing of the input text. >>> For functionality that is implicitly invoked per keystroke, or on >>> mouse hover etc, this difference in efficiency negatively affects >>> the overall user experience of an IDE. >>> >>> >>> (4) The following proposal avoids all difficulties described above: >>> >>> * open, module, requires, transitive, exports, opens, to, uses, >>> provides, and with are "module words", to which the following >>> interpretation is applied: >>> * within any ordinary compilation unit, a module word is a normal >>> identifier. >>> * within a modular compilation unit, all module words are >>> (unconditional) keywords. >>> * We introduce three new auxiliary non-terminals: >>> LegacyPackageName: >>> LegacyIdentifier >>> LegacyPackageName . LegacyIdentifier >>> LegacyTypeName: >>> LegacyIdentifier >>> LegacyTypeName . LegacyIdentifier >>> LegacyIdentifier: >>> Identifier >>> ^open >>> ^module >>> ... >>> ^with >>> * We modify all productions in 7.7, replacing PackageName with >>> LegacyPackageName and replacing TypeName with LegacyTypeName. >>> * After parsing, each of the words '^open', '^module' etc. >>> is interpreted by removing the leading '^' (escape character). >>> >>> Here, '^' is chosen as the escape character following the precedent >>> of Xtext. Plenty of other options for this purpose are possible, too. >>> >>> >>> >>> This proposal completely satisfies the requirements (1), and avoids >>> all of the problems (2) and (3). There's an obvious price to pay: >>> users will have to add the escape character when referring to code >>> that uses a module word as a package name or type name. >>> >>> Not only is this a very low price compared to the benefits; one can >>> even argue that it also helps the human reader of a module declaration, >>> because it clearly marks which occurrences of a module word are indeed >>> identifiers. >>> >>> An IDE can easily help in interactively adding escapes where necessary. >>> >>> Finally, in this trade-off it is relevant to consider the expected >>> frequencies: legacy names (needing escape) will surely be the exception >>> - by magnitudes. So, the little price needing to be paid, will only >>> affect a comparatively small number of locations. >>> >>> >>> Stephan > From michael.rasmussen at zeroturnaround.com Fri May 12 16:18:06 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Fri, 12 May 2017 19:18:06 +0300 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 12 May 2017 at 17:37, Stephen Colebourne wrote: > On 12 May 2017 at 01:43, David M. Lloyd wrote: >> 1. Layer primitive: addExports() - mirrors the existing Module.addExports() >> method for ModuleLayer.Controllers >> 2. Layer primitive: addUses() - mirrors the existing Module.addUses() method >> for ModuleLayer.Controllers >> 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add >> packages to a module after it has been defined > > Are these a good idea? I don't know. What I do know is that the use > cases for them seem to be focussed on low-level coders and framework > writers, which is a very small subset of all Java developers, and a > group who can work around difficulties. > All of these live on ModuleLayer.Controller, a class used for controlling a module layer, that should only be accessible to the code creating that layer. Meaning an API minded exactly for those low-level coders and writers! In the case of addExports and addUses, the functionality is already present through other means, via the corresponding methods on Module, or even via the instrumentation API. And since you are controlling the Layer, you are probably also in control of the classloader(s) in which the modules are defined, meaning it is trivial to define a class into the module to gain access to the methods on Module. Having these methods available on the Controller, seems like a much nicer and cleaner solution, than having to define bridge classes in all modules in order to accomplish it. As for addPackage, as pointed out elsewhere in this thread, it's a bit more tricky... /Michael From david.lloyd at redhat.com Fri May 12 16:31:09 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 11:31:09 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <27effd8b-4914-e33d-60aa-fbd65d51a354@redhat.com> On 05/12/2017 09:37 AM, Stephen Colebourne wrote: > On 12 May 2017 at 01:43, David M. Lloyd wrote: >> 1. Layer primitive: addExports() - mirrors the existing Module.addExports() >> method for ModuleLayer.Controllers >> 2. Layer primitive: addUses() - mirrors the existing Module.addUses() method >> for ModuleLayer.Controllers >> 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add >> packages to a module after it has been defined > > Are these a good idea? I don't know. What I do know is that the use > cases for them seem to be focussed on low-level coders and framework > writers, which is a very small subset of all Java developers, and a > group who can work around difficulties. I appreciate your confidence in our abilities. :) However I think these changes will directly lead to a better user experience for those people who are using such containers. And they don't impact those who don't. A "majority rules" sort of metric doesn't always give a clear idea of cost/benefit, especially when you consider that while most people don't *develop* containers or frameworks, most users of Java *use* them. Frameworks which work smoothly are a key ingredient for apps which work smoothly. >> 4. Make run-time cycle checking optional > > My opinion is that run-time cycles are inevitable. The proposed > solutions (refactoring to API vs Impl) is not particularly good in an > open source context. I'm also concerned that "requires static" > (optional dependencies) also naturally leads to cycles > > But the question at this point is whether it is worth the effort to > try and live without cycles for this release - after all, no-one wants > cycles in their design. It's true, cycles sound bad. But as a point of fact, at run time they're not demonstrably harmful, and in fact worrying overmuch about cycles detracts from clean design principles: I have an ABI that conforms to a particular behavioral contract; it should not matter how I meet that contract (and in particular, whether I do so by way of a series of other contracts that ultimately may be used by things which use *my* ABI is a completely uninteresting distraction). > Since there is a workaround, via the > low-level module/layer code, it feels like we should be willing to > take the punt and try to live without cycles in 9 keeping it under > review to add in 10. What workaround are you referring to? Without this patch, there's no way I can assemble an application with cycles on the module path without the JVM blowing up. If you're referring to the addReads() API, don't think it's fair to users to require that a container or manual module management be employed in these cases, nor to forbid theses users from using jlink. Think about *why* this is being enforced at run time; what benefits will the user receive when they get this error? What about costs? With this change, the cost can be mitigated, while the benefits (whatever they may be) are retained. >> 5. Add optional class loader isolation for modules on the module path > > Again, my opinion is that the isolation of modules is insufficient in > Jigsaw and that hidden packages will sometimes be very messy. However, > the only tool available to solve this appears to be classloaders, and > they have plenty of flaws themselves. While opt-in isolation appears > tempting, I'd prefer to wait and see a better solution in a later > version. Maybe the answer is to deprecated class loaders and come up > with something better! Among the EC members who voted no, this was the most oft-cited technical concern (as far as I could find). I don't think it is realistic to expect a change of vote without addressing the voters' primary concerns; what I'm trying to do is to create a compromise solution. Experience with end user applications running under our modular system and OSGi over the past 5+ years seems to indicate that most software can cope with class loader isolation with few or no changes. I would be highly skeptical that a piece of software being modularized is likely to survive the reflection changes, new service loader behavior, etc., yet somehow fail due to class loader isolation; nevertheless, making it opt-in means that this risk would be minimized. > Given where we are, I think my preference is to see a JDK 9 release > soon rather than end the JSR over these issues. The JSR must be resubmitted within 30 days of the conclusion of the original vote; my hope is to get enough small changes in there that at least the *key* concerns are addressed so that the required 2/3 majority vote is likely. But my purpose of posting these changes here is not to create debate over procedure or politics, but rather to discuss the technical qualities of the patches: Are they acceptable? If not, why not, and what changes would be needed to make them acceptable, or what alternative approaches could solve the same problems in a better way? -- - DML From paul.bakker.nl at gmail.com Fri May 12 16:36:41 2017 From: paul.bakker.nl at gmail.com (Paul Bakker) Date: Fri, 12 May 2017 09:36:41 -0700 Subject: #AddExportsInManifest In-Reply-To: <0d032c3a-d004-a2bc-428e-3dfef8b75193@oracle.com> References: <35A32220-9617-46E7-8DD7-A6910CE434B5@gmail.com> <09301bbd-8728-09aa-9c9e-a9413cbc7741@oracle.com> <9A309FA0-2746-49B0-B4BC-754A68DA4149@gmail.com> <0d032c3a-d004-a2bc-428e-3dfef8b75193@oracle.com> Message-ID: Got it! It makes sense after this example. As a bit of feedback: I do think the naming of the properties is confusing. Because they are the same as the --add-opens/--add-exports flags, my assumption was they would work (exactly) the same. Naming things is hard, and I don't have a better alternative, but maybe a more descriptive name could be considered for the properties? Thanks, Paul > On May 12, 2017, at 12:26 AM, Alan Bateman wrote: > > On 11/05/2017 21:25, Paul Bakker wrote: > >> I'm a little confused by "...deployments work if they are dependent on JDK internal APIs". What does internal JDK usage have to do with opening/exporting your own packages? I would think this solves the problem that some other code (e.g. a library) requires access to application code? >> Also, can you elaborate why this only applies to the unnamed module and can't do the same a the --add-exports/--add-opens flags? >> > Hopefully the mail that Mandy references gives the context. > > As an example, suppose an executable JAR that has code that is hacking into sun.awt.X11 for some reason. To keep that code working then you could do: > > java --add-opens java.desktop/sun.awt.X11=ALL-UNNAMED -jar app.jar > > but this isn't convenient for end-users that double click JAR files. The equivalent for an executable JAR is to add "Add-Opens: java.desktop/sun.awt" to its main manifest. > > -Alan From misterm at gmail.com Fri May 12 16:49:53 2017 From: misterm at gmail.com (Michael Nascimento) Date: Fri, 12 May 2017 13:49:53 -0300 Subject: Some suggested patches and improvements In-Reply-To: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: #1-#3, IMHO, meet the needs of a niche who knows how to work around these issues using other ways (although inconvenient). 4 and 5 are solutions for problems any developer assembling a medium-large application will definitely face and therefore are essential in my point of view since it'd be unreasonable to expect the average developer to master the black magic of low level module APIs or bytecode manipulation. Regards, Michael On Thu, May 11, 2017 at 9:43 PM, David M. Lloyd wrote: > I've proposed five patches to the jpms-spec-experts list [1..5] for > discussion. The patches are as follows: > > 1. Layer primitive: addExports() - mirrors the existing > Module.addExports() method for ModuleLayer.Controllers > 2. Layer primitive: addUses() - mirrors the existing Module.addUses() > method for ModuleLayer.Controllers > 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to add > packages to a module after it has been defined > 4. Make run-time cycle checking optional > 5. Add optional class loader isolation for modules on the module path > > Apart from #3, each of these patches is small and very low-risk, and > address a clear problem. In the case of #3, I believe that the risk is > very low, but am also willing to discuss modifying the patch to move the > method to a "jdk.unsupported" class or in other ways. > > I also am maintaining a mirror of the jigsaw/jake tree as a Git repository > [6] which contains these patches where they can be viewed and fiddled with > on a more interactive basis. > > It is my belief that these small, low-cost patches will go a very long way > towards mitigating many of the problems identified and shared by Red Hat, > by other EG members, by other EC members, and by the community. > > Please have a look and let me know if you have any feedback on any of > these changes. > > [1] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-May/000705.html > [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-May/000707.html > [3] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-May/000708.html > [4] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-May/000711.html > [5] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/201 > 7-May/000710.html > [6] https://github.com/dmlloyd/openjdk-modules > > -- > - DML > From blackdrag at gmx.org Fri May 12 17:21:35 2017 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 12 May 2017 19:21:35 +0200 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <9433d971-e661-5251-3cda-673f43b88ba2@gmx.org> On 12.05.2017 18:49, Michael Nascimento wrote: > #1-#3, IMHO, meet the needs of a niche who knows how to work around these > issues using other ways (although inconvenient). I see me as someone needing this, and only having hints about how to make it work, that are based on annotation processors, which is no acceptable solution at all. bye Jochen From sander.mak at luminis.eu Fri May 12 17:29:29 2017 From: sander.mak at luminis.eu (Sander Mak) Date: Fri, 12 May 2017 17:29:29 +0000 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> <20170505074155.424443679@eggemoggin.niobe.net> Message-ID: <1D4992D4-F8C9-40D9-BB61-E0C939131FC2@luminis.eu> > module guava { > requires transient com.google.common; > } > I suspect you mean 'transitive' here. Sander From scolebourne at joda.org Fri May 12 17:32:01 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 12 May 2017 18:32:01 +0100 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: <1D4992D4-F8C9-40D9-BB61-E0C939131FC2@luminis.eu> References: <20170504103906.689386446@eggemoggin.niobe.net> <20170505074155.424443679@eggemoggin.niobe.net> <1D4992D4-F8C9-40D9-BB61-E0C939131FC2@luminis.eu> Message-ID: On 12 May 2017 at 18:29, Sander Mak wrote: >> module guava { >> requires transient com.google.common; >> } >> > > I suspect you mean 'transitive' here. Yes :-) Stephen From nipa at codefx.org Fri May 12 17:48:28 2017 From: nipa at codefx.org (Nicolai Parlog) Date: Fri, 12 May 2017 19:48:28 +0200 Subject: Revised proposal for #AutomaticModuleNames In-Reply-To: References: <20170504103906.689386446@eggemoggin.niobe.net> <20170505074155.424443679@eggemoggin.niobe.net> Message-ID: Hi Stephen, the shim only works partially. Qualified exports/opens for example will not get past it. I did not yet investigate further to see whether other problems occur. However much we try module names will never be 100% stable and providing an aliasing mechanism would solve this as well as other problems (e.g. dropping in a replacement). But maybe that's something for Java 9.1 or 10. ;) so long ... Nicolai On 12.05.2017 17:23, Stephen Colebourne wrote: > On 5 May 2017 at 15:41, wrote: >> I suspect what you really mean is "Never publish JARs that refer >> to automatic modules that do not have `Automatic-Module-Name` >> attributes". In general that's good advice. It might even be >> reasonable for managers of artifact repositories to insist upon >> it, although I now understand that that could be problematic for >> a repository as popular as Maven Central. > > I think this is the tricky part. Lets review where we are. > > Firstly, we now have super-package reverse-DNS names and MANIFEST > entries. Both of these are a very good thing. > > Secondly, a module with a module-info can now depend on three > things: - an explicit module with a module-info (good) - an > automatic module with MANIFEST entry (good) - an automatic module > with name implied from filename (not good) > > Thirdly, given the nature of Maven Central, it is not realistic to > prevent modules with dependencies on names implied from filenames > getting into Central. > > My remaining concerns with automatic modules are primarily about > this last point - we have to accept that there will be modules with > "bad dependencies" in Maven Central. For example, the graph will > end up with a dependency on "guava" that breaks when Guava is > properly modularized and uses "com.google.common" as the module > name. The result is a graph that does not resolve. While I and > others will try and communicate this issue to the community, I > would prefer that the issue simply could not occur. > > > However, it has been pointed out to me off-list that there is a way > to workaround the name change that I had not fully appreciated > before. > > Given this broken module graph state: > > module com.foo.application { requires guava; } module > com.google.common { } > > it is possible to fix it by introducing an additional module as > follows: > > module guava { requires transient com.google.common; } > > With this additional "renaming shim module" in the graph, > everything now works again. > > Given that there is a potential workaround, I am not as concerned > as I previously was about this issue. > > > Ideally, the creation of this "renaming shim module" should be part > of the JDK (effectively it is a kind of alias, and the JDK is best > placed to manage this). However, it should also be possible for > tools like Maven to provide the shim dynamically when needed. It > would even be possible to release such a shim module to Maven > Central and depend on it in the usual way. > > While this is still a workaround to an issue I wish didn't exist > in the first place, it is a viable solution that I think allows > automatic modules to proceed as is (given the JSR/release timeline > pressure). > > Stephen > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From david.lloyd at redhat.com Fri May 12 18:02:42 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 13:02:42 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <602cc395-058c-31a4-b9ea-46dae4f42a81@redhat.com> On 05/12/2017 08:31 AM, David M. Lloyd wrote: > On 05/12/2017 03:22 AM, Alan Bateman wrote: >> On 12/05/2017 01:43, David M. Lloyd wrote: >> >>> I've proposed five patches to the jpms-spec-experts list [1..5] for >>> discussion. The patches are as follows: >>> [...] >>> 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to >>> add packages to a module after it has been defined [...] >> However for #3 then you've missed several important error cases, e.g. >> illegal package names, or the package is already in another module >> defined to the class loader. There is impact in other areas too. I wasn't worried about validating package names at first, because I figured that you could never define a class to the package anyway, but nevertheless I see the value in error detection (and avoiding piles of cruft inside the JVM regardless of whether it might trip any kind of internal assertion, which I acknowledge as a possibility as well), so I've pushed up a patch [1] to add package name validation. This also detects the empty package name situation. Detecting package conflict is somewhat more subtle; that's coming up next. [1] https://github.com/dmlloyd/openjdk-modules/commit/a636c8891d6958d7b395a57d17c95699ee549500 -- - DML From david.lloyd at redhat.com Fri May 12 18:08:38 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 13:08:38 -0500 Subject: Why not multiple modules of the same name in a class loader? Message-ID: This has come up a couple times now and I'm not sure why the rule exists: why not allow multiple modules with the same name in the same class loader? As far as I can tell, there is no way to locate a module by class loader, and module namespaces are generally already well-governed by their layer, so from an API perspective at least, it seems like this is a harmless scenario (as long as there are no package conflicts, obviously, but a container generally is able to manage this concern). Preventing this does block one potential workaround for the lack of a ModuleLayer.Controller.addPackage() method, which would be to supplement a module's content at run time by defining a new Layer that contains a module of the same name as the original content within the same class loader, where the container would then take care of such details as mutual read/opening and cyclic access. So that made me curious: is there some hotspot-internal dictionary that I missed, which manages the set of modules within a class loader? -- - DML From david.lloyd at redhat.com Fri May 12 18:37:28 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 13:37:28 -0500 Subject: Some suggested patches and improvements In-Reply-To: <602cc395-058c-31a4-b9ea-46dae4f42a81@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> <602cc395-058c-31a4-b9ea-46dae4f42a81@redhat.com> Message-ID: On 05/12/2017 01:02 PM, David M. Lloyd wrote: > On 05/12/2017 08:31 AM, David M. Lloyd wrote: >> On 05/12/2017 03:22 AM, Alan Bateman wrote: >>> On 12/05/2017 01:43, David M. Lloyd wrote: >>> >>>> I've proposed five patches to the jpms-spec-experts list [1..5] for >>>> discussion. The patches are as follows: >>>> > [...] >>>> 3. Layer primitive: addPackage() - allows ModuleLayer.Controllers to >>>> add packages to a module after it has been defined > [...] >>> However for #3 then you've missed several important error cases, e.g. >>> illegal package names, or the package is already in another module >>> defined to the class loader. There is impact in other areas too. > > I wasn't worried about validating package names at first, because I > figured that you could never define a class to the package anyway, but > nevertheless I see the value in error detection (and avoiding piles of > cruft inside the JVM regardless of whether it might trip any kind of > internal assertion, which I acknowledge as a possibility as well), so > I've pushed up a patch [1] to add package name validation. This also > detects the empty package name situation. > > Detecting package conflict is somewhat more subtle; that's coming up next. > > [1] > https://github.com/dmlloyd/openjdk-modules/commit/a636c8891d6958d7b395a57d17c95699ee549500 Here [2] is my first pass at detecting package conflict. As an additional precaution, I forbid packages to be added to the bootstrap class loader in this way. [2] https://github.com/dmlloyd/openjdk-modules/commit/f02719801ac41c6fd9b8c216f16c3f86b909e5fe -- - DML From forax at univ-mlv.fr Fri May 12 18:46:15 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 12 May 2017 18:46:15 +0000 Subject: Why not multiple modules of the same name in a class loader? In-Reply-To: References: Message-ID: <82181055-7D3A-4106-90D9-4E9F84FB51B2@univ-mlv.fr> On May 12, 2017 8:08:38 PM GMT+02:00, "David M. Lloyd" wrote: >This has come up a couple times now and I'm not sure why the rule >exists: why not allow multiple modules with the same name in the same >class loader? module names appear in the stack traces, it will make life of people miserable if they have to open several artifacts to find the good one. and if later the VM is able to isolate un-exported packages, it will be a real mess because at worst, you will have to follow all stack elements to find the class you want. > >As far as I can tell, there is no way to locate a module by class >loader, and module namespaces are generally already well-governed by >their layer, so from an API perspective at least, it seems like this is > >a harmless scenario (as long as there are no package conflicts, >obviously, but a container generally is able to manage this concern). > >Preventing this does block one potential workaround for the lack of a >ModuleLayer.Controller.addPackage() method, which would be to >supplement >a module's content at run time by defining a new Layer that contains a >module of the same name as the original content within the same class >loader, where the container would then take care of such details as >mutual read/opening and cyclic access. > >So that made me curious: is there some hotspot-internal dictionary that > >I missed, which manages the set of modules within a class loader? take a look to defineModule :) Remi -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From david.lloyd at redhat.com Fri May 12 19:09:00 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 14:09:00 -0500 Subject: Why not multiple modules of the same name in a class loader? In-Reply-To: <82181055-7D3A-4106-90D9-4E9F84FB51B2@univ-mlv.fr> References: <82181055-7D3A-4106-90D9-4E9F84FB51B2@univ-mlv.fr> Message-ID: On 05/12/2017 01:46 PM, Remi Forax wrote: > > > On May 12, 2017 8:08:38 PM GMT+02:00, "David M. Lloyd" wrote: >> This has come up a couple times now and I'm not sure why the rule >> exists: why not allow multiple modules with the same name in the same >> class loader? > > module names appear in the stack traces, it will make life of people miserable if they have to open several artifacts to find the good one. I agree; but, I wasn't looking for moral reasons, I was just curious about the technical constraint (if it even exists; maybe it does not!). >> As far as I can tell, there is no way to locate a module by class >> loader, and module namespaces are generally already well-governed by >> their layer, so from an API perspective at least, it seems like this is >> >> a harmless scenario (as long as there are no package conflicts, >> obviously, but a container generally is able to manage this concern). >> >> Preventing this does block one potential workaround for the lack of a >> ModuleLayer.Controller.addPackage() method, which would be to >> supplement >> a module's content at run time by defining a new Layer that contains a >> module of the same name as the original content within the same class >> loader, where the container would then take care of such details as >> mutual read/opening and cyclic access. >> >> So that made me curious: is there some hotspot-internal dictionary that >> >> I missed, which manages the set of modules within a class loader? > > take a look to defineModule :) I assume you mean Module.defineModule0 -> JVM_DefineModule -> Modules::define_module, which is where I looked. It seemed to care about the class loader of the module, and about package names, but unless I'm just being blind to this (always a possibility), I don't see anyplace where it cares about what modules are defined to the class loader. The closest thing I see is the bootstrap code which reads from the jimage, but none of that applies to user ModuleLayers. In other words, I find no Map, or its C++ equivalent, at any level in a class loader that would be capable of such enforcement, let alone require it. I guess I should do some more testing & tracing to see how this part works; maybe it's actually allowed! -- - DML From peter.levart at gmail.com Fri May 12 19:13:37 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 12 May 2017 21:13:37 +0200 Subject: An alternative to "restricted keywords" In-Reply-To: <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> Message-ID: <16708cbd-5005-3c74-dcd3-d9059aebcf63@gmail.com> On 05/12/2017 06:08 PM, Peter Levart wrote: >> For me, a restricted keyword is a keyword which is activated if you >> are at a position in the grammar where it can be recognized and >> because it's a keyword, it tooks over an identifier. >> by example for >> module m { >> if the next token is 'requires', it should be recognized as a keyword >> because you can parse a directive 'required ...' so there is a >> production that will starts with the 'required' keyword. >> >> so >> module m { requires transitive; } >> should be rejected because transitive should be recognized as a >> keyword after requires and the compiler should report a missing >> module name. >> and >> module m { requires transitive transitive; } >> should be rejected because the grammar that parse the modifiers is >> defined as "a loop" so from the grammar point of view it's like >> module m { requires Modifier Modifier; } >> so the the front end of the compiler should report a missing module >> name and a later phase should report that there is twice the same >> modifier 'transitive'. >> >> I believe that with this definition of 'restricted keyword', compiler >> can recover error more easily and offers meaningful error message and >> the module-info part of the grammar is LR(1). > > This will make "requires", "uses", "provides", "with", "to", "static", > "transitive", "exports", etc .... all illegal module names. Ok, no big > deal, because there are no module names yet (apart from JDK modules > and those are named differently). But... > > What about: > > module m { exports transitive; } ...ok, I realized there's no problem in exports or opens as there are no exports/opens modifiers in the current syntax, so what follows 'exports' or 'opens' can always be interpreted as package name only. But what if some future extension of the language wants to define an exports or opens modifier? Or some new yet unexistent requires modifier? Peter From forax at univ-mlv.fr Fri May 12 19:21:19 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 12 May 2017 19:21:19 +0000 Subject: An alternative to "restricted keywords" In-Reply-To: <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> Message-ID: <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> Hi Peter, On May 12, 2017 6:08:58 PM GMT+02:00, Peter Levart wrote: >Hi Remi, > >On 05/12/2017 08:17 AM, Remi Forax wrote: >> [CC JPMS expert mailing list because, it's an important issue IMO] >> >> I've a counter proposition. >> >> I do not like your proposal because from the user point of view, '^' >looks like a hack, it's not used anywhere else in the grammar. >> I agree that restricted keywords are not properly specified in JLS. >Reading your mail, i've discovered that what i was calling restricted >keywords is not what javac implements :( >> I agree that restricted keywords should be only enabled when parsing >module-info.java >> I agree that doing error recovery on the way the grammar for >module-info is currently implemented in javac leads to less than ideal >error messages. >> >> In my opinion, both >> module m { requires transitive transitive; } >> module m { requires transitive; } >> should be rejected because what javac implements something more close >to the javascript ASI rules than restricted keywords as currently >specified by Alex. >> >> For me, a restricted keyword is a keyword which is activated if you >are at a position in the grammar where it can be recognized and because >it's a keyword, it tooks over an identifier. >> by example for >> module m { >> if the next token is 'requires', it should be recognized as a keyword >because you can parse a directive 'required ...' so there is a >production that will starts with the 'required' keyword. >> >> so >> module m { requires transitive; } >> should be rejected because transitive should be recognized as a >keyword after requires and the compiler should report a missing module >name. >> >> and >> module m { requires transitive transitive; } >> should be rejected because the grammar that parse the modifiers is >defined as "a loop" so from the grammar point of view it's like >> module m { requires Modifier Modifier; } >> so the the front end of the compiler should report a missing module >name and a later phase should report that there is twice the same >modifier 'transitive'. >> >> I believe that with this definition of 'restricted keyword', compiler >can recover error more easily and offers meaningful error message and >the module-info part of the grammar is LR(1). > >This will make "requires", "uses", "provides", "with", "to", "static", >"transitive", "exports", etc .... all illegal module names. Ok, no big >deal, because there are no module names yet (apart from JDK modules and > >those are named differently). But... you should use reverse DNS naming for module name, so no problem. > >What about: > >module m { exports transitive; } > >Here 'transitive' is an existing package name for example. Who >guarantees that there are no packages out there with names matching >restricted keywords? Current restriction for modules is that they can >not have an unnamed package. Do we want to restrict package names a >module can export too? you should use reverse DNS naming for package so no problem :) > >Stephan's solution does not have this problem. > >Regards, Peter I think those issues are not real problem. R?mi > >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "Stephan Herrmann" >>> ?: jigsaw-dev at openjdk.java.net >>> Envoy?: Mardi 9 Mai 2017 16:56:11 >>> Objet: An alternative to "restricted keywords" >>> (1) I understand the need for avoiding that new module-related >>> keywords conflict with existing code, where these words may be used >>> as identifiers. Moreover, it must be possible for a module >declaration >>> to refer to packages or types thusly named. >>> >>> However, >>> >>> (2) The currently proposed "restricted keywords" are not >appropriately >>> specified in JLS. >>> >>> (3) The currently proposed "restricted keywords" pose difficulties >to >>> the implementation of all tools that need to parse a module >declaration. >>> >>> (4) A simple alternative to "restricted keywords" exists, which has >not >>> received the attention it deserves. >>> >>> Details: >>> >>> (2) The current specification implicitly violates the assumption >that >>> parsing can be performed on the basis of a token stream produced by >>> a scanner (aka lexer). From discussion on this list we learned that >>> the following examples are intended to be syntactically legal: >>> module m { requires transitive transitive; } >>> module m { requires transitive; } >>> (Please for the moment disregard heuristic solutions, while we are >>> investigating whether generally "restricted keywords" is a >well-defined >>> concept, or not.) >>> Of the three occurrences of "transitive", #1 is a keyword, the >others >>> are identifiers. At the point when the parser has consumed >"requires" >>> and now asks about classification of the word "transitive", the >scanner >>> cannot possible answer this classification. It can only answer for >sure, >>> after the *parser* has accepted the full declaration. Put >differently, >>> the parser must consume more tokens than have been classified by the >>> Scanner. Put differently, to faithfully parse arbitrary grammars >using >>> a concept of "restricted keywords", scanners must provide >speculative >>> answers, which may later need to be revised by backtracking or >similar >>> exhaustive exploration of the space of possible interpretations. >>> >>> The specification is totally silent about this fundamental change. >>> >>> >>> (3) "restricted keywords" pose three problems to tool >implementations: >>> >>> (3.a) Any known practical approach to implement a parser with >>> "restricted keywords" requires to leverage heuristics, which are >based >>> on the exact set of rules defined in the grammar. Such heuristics >>> reduce the look-ahead that needs to be performed by the scanner, >>> in order to avoid the full exhaustive exploration mentioned above. >>> A set of such heuristic is extremely fragile and can easily break >when >>> later more rules are added to the grammar. This means small future >>> language changes can easily break any chosen strategy. >>> >>> (3.b) If parsing works for error-free input, this doesn't imply that >>> a parser will be able to give any useful answer for input with >syntax >>> errors. As a worst-case example consider an arbitrary input sequence >>> consisting of just the two words "requires" and "transitive" in >random >>> order and with no punctuation. >>> A parser will not be able to detect any structure in this sequence. >>> By comparison, normal keywords serve as a baseline, where parsing >>> typically can resume regardless of any leading garbage. >>> While this is not relevant for normal compilation, it is paramount >>> for assistive functions, which most of the time operate on >incomplete >>> text, likely to contain even syntax errors. >>> Strictly speaking, any "module declaration" with syntax errors is >>> not a ModuleDeclaration, and thus none of the "restrictive keywords" >>> can be interpreted as keywords (which per JLS can only happen inside >>> a ModuleDeclaration). >>> All this means, that functionality like code completion is >>> systematically broken in a language using "restricted keywords". >>> >>> (3.c) Other IDE functionality assumes that small fragments of the >>> input text can be scanned out of context. The classical example here >>> is syntax highlighting but there are more examples. >>> Any such functionality has to be re-implemented, replacing the >>> highly efficient local scanning with full parsing of the input text. >>> For functionality that is implicitly invoked per keystroke, or on >>> mouse hover etc, this difference in efficiency negatively affects >>> the overall user experience of an IDE. >>> >>> >>> (4) The following proposal avoids all difficulties described above: >>> >>> * open, module, requires, transitive, exports, opens, to, uses, >>> provides, and with are "module words", to which the following >>> interpretation is applied: >>> * within any ordinary compilation unit, a module word is a >normal >>> identifier. >>> * within a modular compilation unit, all module words are >>> (unconditional) keywords. >>> * We introduce three new auxiliary non-terminals: >>> LegacyPackageName: >>> LegacyIdentifier >>> LegacyPackageName . LegacyIdentifier >>> LegacyTypeName: >>> LegacyIdentifier >>> LegacyTypeName . LegacyIdentifier >>> LegacyIdentifier: >>> Identifier >>> ^open >>> ^module >>> ... >>> ^with >>> * We modify all productions in 7.7, replacing PackageName with >>> LegacyPackageName and replacing TypeName with LegacyTypeName. >>> * After parsing, each of the words '^open', '^module' etc. >>> is interpreted by removing the leading '^' (escape character). >>> >>> Here, '^' is chosen as the escape character following the precedent >>> of Xtext. Plenty of other options for this purpose are possible, >too. >>> >>> >>> >>> This proposal completely satisfies the requirements (1), and avoids >>> all of the problems (2) and (3). There's an obvious price to pay: >>> users will have to add the escape character when referring to code >>> that uses a module word as a package name or type name. >>> >>> Not only is this a very low price compared to the benefits; one can >>> even argue that it also helps the human reader of a module >declaration, >>> because it clearly marks which occurrences of a module word are >indeed >>> identifiers. >>> >>> An IDE can easily help in interactively adding escapes where >necessary. >>> >>> Finally, in this trade-off it is relevant to consider the expected >>> frequencies: legacy names (needing escape) will surely be the >exception >>> - by magnitudes. So, the little price needing to be paid, will only >>> affect a comparatively small number of locations. >>> >>> >>> Stephan -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From david.lloyd at redhat.com Fri May 12 19:40:38 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 May 2017 14:40:38 -0500 Subject: Why not multiple modules of the same name in a class loader? In-Reply-To: References: <82181055-7D3A-4106-90D9-4E9F84FB51B2@univ-mlv.fr> Message-ID: On 05/12/2017 02:09 PM, David M. Lloyd wrote: > On 05/12/2017 01:46 PM, Remi Forax wrote: >> >> >> On May 12, 2017 8:08:38 PM GMT+02:00, "David M. Lloyd" >> wrote: >>> This has come up a couple times now and I'm not sure why the rule >>> exists: why not allow multiple modules with the same name in the same >>> class loader? >> >> module names appear in the stack traces, it will make life of people >> miserable if they have to open several artifacts to find the good one. > > I agree; but, I wasn't looking for moral reasons, I was just curious > about the technical constraint (if it even exists; maybe it does not!). > >>> As far as I can tell, there is no way to locate a module by class >>> loader, and module namespaces are generally already well-governed by >>> their layer, so from an API perspective at least, it seems like this is >>> >>> a harmless scenario (as long as there are no package conflicts, >>> obviously, but a container generally is able to manage this concern). >>> >>> Preventing this does block one potential workaround for the lack of a >>> ModuleLayer.Controller.addPackage() method, which would be to >>> supplement >>> a module's content at run time by defining a new Layer that contains a >>> module of the same name as the original content within the same class >>> loader, where the container would then take care of such details as >>> mutual read/opening and cyclic access. >>> >>> So that made me curious: is there some hotspot-internal dictionary that >>> >>> I missed, which manages the set of modules within a class loader? >> >> take a look to defineModule :) > > I assume you mean Module.defineModule0 -> JVM_DefineModule -> > Modules::define_module, which is where I looked. It seemed to care > about the class loader of the module, and about package names, but > unless I'm just being blind to this (always a possibility), I don't see > anyplace where it cares about what modules are defined to the class > loader. The closest thing I see is the bootstrap code which reads from > the jimage, but none of that applies to user ModuleLayers. > > In other words, I find no Map, or its C++ equivalent, at > any level in a class loader that would be capable of such enforcement, > let alone require it. > > I guess I should do some more testing & tracing to see how this part > works; maybe it's actually allowed! Found it: there's a module table on ClassLoaderData that you get via its modules() method. -- - DML From michael.rasmussen at zeroturnaround.com Fri May 12 20:52:56 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Fri, 12 May 2017 23:52:56 +0300 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 12 May 2017 at 11:22, Alan Bateman wrote: > However for #3 then you've > missed several important error cases, e.g. illegal package names, or the > package is already in another module defined to the class loader. These checks are already present in implAddPackage, so why duplicate those checks in the wrapping method? Speaking of, the package name check in implAddPackage seems incomplete. addPackage("some...pkg") would pass the check, but it's shouldn't be legal, as it contains empty segments. /Michael From sander.mak at luminis.eu Mon May 15 19:43:21 2017 From: sander.mak at luminis.eu (Sander Mak) Date: Mon, 15 May 2017 19:43:21 +0000 Subject: jdeps --generate-open-module Message-ID: <537D2161-CEF3-4802-A14A-9969E64A21F1@luminis.eu> When executing `jdeps --generate-module-info ./out `, a module descriptor exporting all packages is created. When using `--generate-open-module`, a module descriptor with an open module is generated, but no packages are exported. I would expect to be able to generate a module descriptor that is as close to the automatic module behavior of the JAR in question, as starting point for modularization. There seems to be no way to generate an open module exporting all packages, which is the closest to automatic module behavior as possible. What's the intended use-case of `--generate-open-module` as it stands? Sander From reto.merz at abacus.ch Mon May 15 14:10:50 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Mon, 15 May 2017 16:10:50 +0200 Subject: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service Message-ID: Hello, It seems that Java 9 Webstart (still) does not add all modules to the class/module path. Our Webstart app is signed, requests all-permision and works with the latest Java 6, 7 and 8. With 9-ea+169 we get this error: java.lang.NoClassDefFoundError: javax/xml/ws/Service ... first 45 from cause Caused by: java.lang.ClassNotFoundException: javax.xml.ws.Service at jdk.deploy at 9-ea/com.sun.deploy.security.DeployURLClassLoader.findClass(Unknown Source) at jdk.javaws at 9-ea/com.sun.jnlp.JNLPClassLoader.findClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at jdk.javaws at 9-ea/com.sun.jnlp.JNLPClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.defineClass1(Native Method) at java.base/java.lang.ClassLoader.defineClass(Unknown Source) at java.base/java.security.SecureClassLoader.defineClass(Unknown Source) at jdk.deploy at 9-ea/com.sun.deploy.security.DeployURLClassLoader.defineClass(Unknown Source) at jdk.deploy at 9-ea/com.sun.deploy.security.DeployURLClassLoader$1.run(Unknown Source) ... 1 recursions at java.base/java.security.AccessController.doPrivileged(Native Method) at jdk.deploy at 9-ea/com.sun.deploy.security.DeployURLClassLoader.findClass(Unknown Source) at jdk.javaws at 9-ea/com.sun.jnlp.JNLPClassLoader.findClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at jdk.javaws at 9-ea/com.sun.jnlp.JNLPClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Unknown Source) Is this a known bug or should this be reported via bugreport.java.com? Thanks Reto From Alan.Bateman at oracle.com Tue May 16 06:00:21 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 07:00:21 +0100 Subject: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: References: Message-ID: On 15/05/2017 15:10, Reto Merz wrote: > Hello, > > It seems that Java 9 Webstart (still) does not add all modules to the class/module path. > Our Webstart app is signed, requests all-permision and works with the latest Java 6, 7 and 8. > > With 9-ea+169 we get this error: > > java.lang.NoClassDefFoundError: javax/xml/ws/Service > ... first 45 from cause > Caused by: java.lang.ClassNotFoundException: javax.xml.ws.Service > at jdk.deploy at 9-ea/com.sun.deploy.security.DeployURLClassLoader.findClass(Unknown Source) > : > > Is this a known bug or should this be reported via bugreport.java.com? > Java Web Start is aligned with the default policy for root modules in JEP 261. This means that the EE modules aren't resolved by default. I suspect this is what you need in the JNLP: -Alan From reto.merz at abacus.ch Tue May 16 08:49:30 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Tue, 16 May 2017 10:49:30 +0200 Subject: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service Message-ID: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> Thanks Alan. Wouldn't it make sense to add all modules by default in Java 9 in case of Webstart and print a warning to the Webstart console the first time a future non-default module is accessed with the message that the JNLP must be extended with a --add-modules=moduleXY in Java 10? So all the Webstart app devs would get more time to adjust the JNLP to support Java9/10. Regards Reto > -----Urspr?ngliche Nachricht----- > Von: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Gesendet: Dienstag, 16. Mai 2017 08:00 > An: Reto Merz; jigsaw-dev at openjdk.java.net > Betreff: Re: Java 9 EA 169 Webstart NoClassDefFoundError: > javax/xml/ws/Service > > On 15/05/2017 15:10, Reto Merz wrote: > > > Hello, > > > > It seems that Java 9 Webstart (still) does not add all modules to the > class/module path. > > Our Webstart app is signed, requests all-permision and works with the > latest Java 6, 7 and 8. > > > > With 9-ea+169 we get this error: > > > > java.lang.NoClassDefFoundError: javax/xml/ws/Service > > ... first 45 from cause > > Caused by: java.lang.ClassNotFoundException: javax.xml.ws.Service > > at jdk.deploy at 9- > ea/com.sun.deploy.security.DeployURLClassLoader.findClass(Unknown > Source) > > : > > > > Is this a known bug or should this be reported via bugreport.java.com? > > > Java Web Start is aligned with the default policy for root modules in > JEP 261. This means that the EE modules aren't resolved by default. I > suspect this is what you need in the JNLP: > > > > -Alan From peter.levart at gmail.com Tue May 16 08:50:21 2017 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 16 May 2017 10:50:21 +0200 Subject: What happened to ModuleClassLoader? In-Reply-To: <09854caf-d3aa-ecce-1a72-701b3cf9e2b3@gmail.com> References: <09854caf-d3aa-ecce-1a72-701b3cf9e2b3@gmail.com> Message-ID: <5aef8876-5eae-061f-d5b1-d83f356be882@gmail.com> Re-sending since I don't see my previous post in the jigsaw-dev archives due to problems with mailing list in the past days... -------- Forwarded Message -------- Subject: What happened to ModuleClassLoader Date: Sun, 14 May 2017 23:11:37 +0200 From: Peter Levart To: jigsaw-dev Hi, If I remember correctly, there was a phase in jigsaw development when there was a class named ModuleClassLoader. It has since disappeared. There are static and instance methods in ModuleLayer: defineModulesWithOneLoader defineModulesWithManyLoaders ...which create appropriate ClassLoader(s) that know how to load classes and resources from modules defined in the Configuration. But what about method: defineModules ...which takes a classloader mapping function (Function). I can't find helpers for constructing appropriate ClassLoader(s) that know how to load classes and resources from the modules defined in the Configuration. What do you recommend? Should user look into jdk.internal.loader.Loader implementation to figure out the details? Suppose that user wants to create a layer that assigns class loaders to modules in a manner that is neither 1 ClassLoader for all modules nor 1 ClassLoader per module. He doesn't want to implement details and just wants to partition modules into disjoint subsets and assign each subset a separate ClassLoader which would be provided by the platform. Something that could be specified in a method with the following signature: defineModulesWithLoaderPerSubset(Configuration cf, List parentLayers, Function msskf); Where: msskf - The function to map a module name to a subset key (a key is an object with appropriate equals/hashCode method - modules mapping to equal keys belong to the same subset. each subset gets a separate ClassLoader provided by the platform) The method could check that module read edges and partitioning into a subset(s) don't conflict and arrange delegation among class loaders appropriately. Above two utility methods (defineModulesWithOneLoader and defineModulesWithManyLoaders) could then be just special cases: defineModulesWithLoaderPerSubset(..., mn -> true); // equivalent to defineModulesWithOneLoader(...) defineModulesWithLoaderPerSubset(..., mn -> mn); // equivalent to defineModulesWithManyLoaders(...) What do you think? Regards, Peter From stephan.herrmann at berlin.de Tue May 16 09:49:45 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Tue, 16 May 2017 11:49:45 +0200 Subject: An alternative to "restricted keywords" In-Reply-To: <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> Message-ID: <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> Thanks, Remi, for taking this to the EG list. Some collected responses: Remi: "from the user point of view, '^' looks like a hack" This is, of course, a subjective statement. I don't share this view and in years of experience with Xtext-languages (where this concept is used by default) I never heard any user complain about this. More importantly, I hold that such aesthetic considerations are of much lesser significance than the question, whether we can explain - unambiguously explain - the concept in a few simple sentences. Explaining must be possible at two levels: in a rigorous specification and in simple words for users of the language. Remi: "a keyword which is activated if you are at a position in the grammar where it can be recognized". I don't think 'being at a position in the grammar' is a good way of explaining. Parsing doesn't generally have one position in a grammar, multiple productions can be active in the same parser state. Also speaking of a "loop" for modifiers seems to complicate matters more than necessary. Under these considerations I still see '^' as the clearest of all solutions. Clear as a specification, simple to explain to users. Peter spoke about module names vs. package names. I think we agree, that module names cannot use "module words", whereas package names should be expected to contain them. Remi: "you should use reverse DNS naming for package so no problem :)" "to" is a "module word" and a TLD. I think we should be very careful in judging that a existing conflict is not a real problem. Better to clearly and rigorously avoid the conflict in the first place. Some additional notes from my side: In the escape-approach, it may be prudent to technically allow escaping even words that are identifiers in Java 9, but could become keywords in a future version. This ensures that modules which need more escaping in Java 9+X can still be parsed in Java 9. Current focus was on names of modules, packages and types. A complete solution must also give an answer for annotations on modules. Some possible solutions: a. Assume that annotations for modules are designed with modules in mind and thus have to avoid any module words in their names. b. Support escaping also in annotations c. Refine the scope where "module words" are keywords, let it start only when the word "module" or the group "open module" has been consumed. This would make the words "module" and "open" special, as being switch words, where we switch from one language to another. (For this I previously coined the term "scoped keywords" [1]) I think we all agree that the conflicts we are solving here are rare corner cases. Most names do not contain module words. Still, from a conceptual and technical p.o.v. the solution must be bullet proof. But there's no need to be afraid of module declarations being spammed with dozens of '^' characters. Realistically, this will not happen. Stephan [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 On 12.05.2017 21:21, Remi Forax wrote: > Hi Peter, > > On May 12, 2017 6:08:58 PM GMT+02:00, Peter Levart wrote: >> Hi Remi, >> >> On 05/12/2017 08:17 AM, Remi Forax wrote: >>> [CC JPMS expert mailing list because, it's an important issue IMO] >>> >>> I've a counter proposition. >>> >>> I do not like your proposal because from the user point of view, '^' >> looks like a hack, it's not used anywhere else in the grammar. >>> I agree that restricted keywords are not properly specified in JLS. >> Reading your mail, i've discovered that what i was calling restricted >> keywords is not what javac implements :( >>> I agree that restricted keywords should be only enabled when parsing >> module-info.java >>> I agree that doing error recovery on the way the grammar for >> module-info is currently implemented in javac leads to less than ideal >> error messages. >>> >>> In my opinion, both >>> module m { requires transitive transitive; } >>> module m { requires transitive; } >>> should be rejected because what javac implements something more close >> to the javascript ASI rules than restricted keywords as currently >> specified by Alex. >>> >>> For me, a restricted keyword is a keyword which is activated if you >> are at a position in the grammar where it can be recognized and because >> it's a keyword, it tooks over an identifier. >>> by example for >>> module m { >>> if the next token is 'requires', it should be recognized as a keyword >> because you can parse a directive 'required ...' so there is a >> production that will starts with the 'required' keyword. >>> >>> so >>> module m { requires transitive; } >>> should be rejected because transitive should be recognized as a >> keyword after requires and the compiler should report a missing module >> name. >>> >>> and >>> module m { requires transitive transitive; } >>> should be rejected because the grammar that parse the modifiers is >> defined as "a loop" so from the grammar point of view it's like >>> module m { requires Modifier Modifier; } >>> so the the front end of the compiler should report a missing module >> name and a later phase should report that there is twice the same >> modifier 'transitive'. >>> >>> I believe that with this definition of 'restricted keyword', compiler >> can recover error more easily and offers meaningful error message and >> the module-info part of the grammar is LR(1). >> >> This will make "requires", "uses", "provides", "with", "to", "static", >> "transitive", "exports", etc .... all illegal module names. Ok, no big >> deal, because there are no module names yet (apart from JDK modules and >> >> those are named differently). But... > > you should use reverse DNS naming for module name, so no problem. > >> >> What about: >> >> module m { exports transitive; } >> >> Here 'transitive' is an existing package name for example. Who >> guarantees that there are no packages out there with names matching >> restricted keywords? Current restriction for modules is that they can >> not have an unnamed package. Do we want to restrict package names a >> module can export too? > > you should use reverse DNS naming for package so no problem :) > >> >> Stephan's solution does not have this problem. >> >> Regards, Peter > > I think those issues are not real problem. > > R?mi > >> >>> >>> regards, >>> R?mi >>> >>> ----- Mail original ----- >>>> De: "Stephan Herrmann" >>>> ?: jigsaw-dev at openjdk.java.net >>>> Envoy?: Mardi 9 Mai 2017 16:56:11 >>>> Objet: An alternative to "restricted keywords" >>>> (1) I understand the need for avoiding that new module-related >>>> keywords conflict with existing code, where these words may be used >>>> as identifiers. Moreover, it must be possible for a module >> declaration >>>> to refer to packages or types thusly named. >>>> >>>> However, >>>> >>>> (2) The currently proposed "restricted keywords" are not >> appropriately >>>> specified in JLS. >>>> >>>> (3) The currently proposed "restricted keywords" pose difficulties >> to >>>> the implementation of all tools that need to parse a module >> declaration. >>>> >>>> (4) A simple alternative to "restricted keywords" exists, which has >> not >>>> received the attention it deserves. >>>> >>>> Details: >>>> >>>> (2) The current specification implicitly violates the assumption >> that >>>> parsing can be performed on the basis of a token stream produced by >>>> a scanner (aka lexer). From discussion on this list we learned that >>>> the following examples are intended to be syntactically legal: >>>> module m { requires transitive transitive; } >>>> module m { requires transitive; } >>>> (Please for the moment disregard heuristic solutions, while we are >>>> investigating whether generally "restricted keywords" is a >> well-defined >>>> concept, or not.) >>>> Of the three occurrences of "transitive", #1 is a keyword, the >> others >>>> are identifiers. At the point when the parser has consumed >> "requires" >>>> and now asks about classification of the word "transitive", the >> scanner >>>> cannot possible answer this classification. It can only answer for >> sure, >>>> after the *parser* has accepted the full declaration. Put >> differently, >>>> the parser must consume more tokens than have been classified by the >>>> Scanner. Put differently, to faithfully parse arbitrary grammars >> using >>>> a concept of "restricted keywords", scanners must provide >> speculative >>>> answers, which may later need to be revised by backtracking or >> similar >>>> exhaustive exploration of the space of possible interpretations. >>>> >>>> The specification is totally silent about this fundamental change. >>>> >>>> >>>> (3) "restricted keywords" pose three problems to tool >> implementations: >>>> >>>> (3.a) Any known practical approach to implement a parser with >>>> "restricted keywords" requires to leverage heuristics, which are >> based >>>> on the exact set of rules defined in the grammar. Such heuristics >>>> reduce the look-ahead that needs to be performed by the scanner, >>>> in order to avoid the full exhaustive exploration mentioned above. >>>> A set of such heuristic is extremely fragile and can easily break >> when >>>> later more rules are added to the grammar. This means small future >>>> language changes can easily break any chosen strategy. >>>> >>>> (3.b) If parsing works for error-free input, this doesn't imply that >>>> a parser will be able to give any useful answer for input with >> syntax >>>> errors. As a worst-case example consider an arbitrary input sequence >>>> consisting of just the two words "requires" and "transitive" in >> random >>>> order and with no punctuation. >>>> A parser will not be able to detect any structure in this sequence. >>>> By comparison, normal keywords serve as a baseline, where parsing >>>> typically can resume regardless of any leading garbage. >>>> While this is not relevant for normal compilation, it is paramount >>>> for assistive functions, which most of the time operate on >> incomplete >>>> text, likely to contain even syntax errors. >>>> Strictly speaking, any "module declaration" with syntax errors is >>>> not a ModuleDeclaration, and thus none of the "restrictive keywords" >>>> can be interpreted as keywords (which per JLS can only happen inside >>>> a ModuleDeclaration). >>>> All this means, that functionality like code completion is >>>> systematically broken in a language using "restricted keywords". >>>> >>>> (3.c) Other IDE functionality assumes that small fragments of the >>>> input text can be scanned out of context. The classical example here >>>> is syntax highlighting but there are more examples. >>>> Any such functionality has to be re-implemented, replacing the >>>> highly efficient local scanning with full parsing of the input text. >>>> For functionality that is implicitly invoked per keystroke, or on >>>> mouse hover etc, this difference in efficiency negatively affects >>>> the overall user experience of an IDE. >>>> >>>> >>>> (4) The following proposal avoids all difficulties described above: >>>> >>>> * open, module, requires, transitive, exports, opens, to, uses, >>>> provides, and with are "module words", to which the following >>>> interpretation is applied: >>>> * within any ordinary compilation unit, a module word is a >> normal >>>> identifier. >>>> * within a modular compilation unit, all module words are >>>> (unconditional) keywords. >>>> * We introduce three new auxiliary non-terminals: >>>> LegacyPackageName: >>>> LegacyIdentifier >>>> LegacyPackageName . LegacyIdentifier >>>> LegacyTypeName: >>>> LegacyIdentifier >>>> LegacyTypeName . LegacyIdentifier >>>> LegacyIdentifier: >>>> Identifier >>>> ^open >>>> ^module >>>> ... >>>> ^with >>>> * We modify all productions in 7.7, replacing PackageName with >>>> LegacyPackageName and replacing TypeName with LegacyTypeName. >>>> * After parsing, each of the words '^open', '^module' etc. >>>> is interpreted by removing the leading '^' (escape character). >>>> >>>> Here, '^' is chosen as the escape character following the precedent >>>> of Xtext. Plenty of other options for this purpose are possible, >> too. >>>> >>>> >>>> >>>> This proposal completely satisfies the requirements (1), and avoids >>>> all of the problems (2) and (3). There's an obvious price to pay: >>>> users will have to add the escape character when referring to code >>>> that uses a module word as a package name or type name. >>>> >>>> Not only is this a very low price compared to the benefits; one can >>>> even argue that it also helps the human reader of a module >> declaration, >>>> because it clearly marks which occurrences of a module word are >> indeed >>>> identifiers. >>>> >>>> An IDE can easily help in interactively adding escapes where >> necessary. >>>> >>>> Finally, in this trade-off it is relevant to consider the expected >>>> frequencies: legacy names (needing escape) will surely be the >> exception >>>> - by magnitudes. So, the little price needing to be paid, will only >>>> affect a comparatively small number of locations. >>>> >>>> >>>> Stephan > From Alan.Bateman at oracle.com Tue May 16 10:06:46 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 11:06:46 +0100 Subject: What happened to ModuleClassLoader? In-Reply-To: <5aef8876-5eae-061f-d5b1-d83f356be882@gmail.com> References: <09854caf-d3aa-ecce-1a72-701b3cf9e2b3@gmail.com> <5aef8876-5eae-061f-d5b1-d83f356be882@gmail.com> Message-ID: On 16/05/2017 09:50, Peter Levart wrote: > : > > If I remember correctly, there was a phase in jigsaw development when > there was a class named ModuleClassLoader. There was although it was replaced in 2015. > : > > Above two utility methods (defineModulesWithOneLoader and > defineModulesWithManyLoaders) could then be just special cases: > > defineModulesWithLoaderPerSubset(..., mn -> true); // equivalent to > defineModulesWithOneLoader(...) > defineModulesWithLoaderPerSubset(..., mn -> mn); // equivalent to > defineModulesWithManyLoaders(...) > > What do you think? My personal view is that we should keep the complexity out of the API. If super advanced users want to use their own class loaders to load modules then they can do so with the defineModules. Yes, this does mean that anyone doing this needs to know how to translate readability and exports into delegation but it's a small/advanced group I think. -Alan From Alan.Bateman at oracle.com Tue May 16 10:14:37 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 11:14:37 +0100 Subject: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> Message-ID: <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> On 16/05/2017 09:49, Reto Merz wrote: > Thanks Alan. > > Wouldn't it make sense to add all modules by default in Java 9 in case of Webstart and print > a warning to the Webstart console the first time a future non-default module is accessed > with the message that the JNLP must be extended with a --add-modules=moduleXY in Java 10? > > So all the Webstart app devs would get more time to adjust the JNLP to support Java9/10. > The proposal in JSR 379 is java.xml.ws and the other modules that overlap with EE be deprecated (forRemoval=true) so that we can drop them from Java SE in a future release (hopefully 10 if the JSR for Java SE 10 agrees). So I think Java Web Start is right to align with the policy for applications launched on the command line. -Alan. From scolebourne at joda.org Tue May 16 10:14:31 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 16 May 2017 11:14:31 +0100 Subject: Some suggested patches and improvements In-Reply-To: <27effd8b-4914-e33d-60aa-fbd65d51a354@redhat.com> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> <27effd8b-4914-e33d-60aa-fbd65d51a354@redhat.com> Message-ID: On 12 May 2017 at 17:31, David M. Lloyd wrote: >>> 4. Make run-time cycle checking optional >> >> My opinion is that run-time cycles are inevitable. The proposed >> solutions (refactoring to API vs Impl) is not particularly good in an >> open source context. I'm also concerned that "requires static" >> (optional dependencies) also naturally leads to cycles >> >> But the question at this point is whether it is worth the effort to >> try and live without cycles for this release - after all, no-one wants >> cycles in their design. > > It's true, cycles sound bad. But as a point of fact, at run time they're > not demonstrably harmful, and in fact worrying overmuch about cycles > detracts from clean design principles: I have an ABI that conforms to a > particular behavioral contract; it should not matter how I meet that > contract (and in particular, whether I do so by way of a series of other > contracts that ultimately may be used by things which use *my* ABI is a > completely uninteresting distraction). Just to be clear, of the five items listed, I think if just one were to be tackled, I'd choose this one. Because I think it will come up in real systems and because I basically agree that cycles at runtime should not be something that should be blocked by the JDK/JVM. Stephen From Alan.Bateman at oracle.com Tue May 16 10:39:26 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 11:39:26 +0100 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <09d65243-c7e7-cf3a-393d-b4ac5eca1989@oracle.com> On 12/05/2017 14:31, David M. Lloyd wrote: > : > >> #4 seems to be working around the outcome of issue #CyclicDependences >> in the JSR. I also don't wish to comment on that except to say that >> introducing system properties to skip specified checks is highly >> problematic from a conformance perspective. > > Can you explain what you mean by that? I'm more than happy to convert > these into -X type arguments to the runtime (or, of course, to simply > make this the standard behavior), but I thought this would be a > simpler and safer approach (at least for a first pass). > Disabling specified behavior creates a conformance issue. It doesn't matter if it's a CLI option or a system property. In any case, proposing a patch here is not the right way to re-open JSR issue #CyclicDependences. Instead, I think it would be better to work through a number of migration scenarios and see what real-issues you run into. Do you run into cycles that can't be addressed with clean-up and migrating to services for examples? -Alan. From Alan.Bateman at oracle.com Tue May 16 11:02:32 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 12:02:32 +0100 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 12/05/2017 14:31, David M. Lloyd wrote: > : > >> There is a lot more to #5, something that will become clear when you >> work through all the scenarios. The JSR and spec part are minor >> though but I'd prefer to hold off until there is more discussion on >> this topic in the JSR. > > I'd rather not hold off as the JSR essentially only has a couple of > weeks left to live if there is not a revised PR. Could you please > explain what you mean? Are you referring to jlink, jaotc, or > something else? > For the most part, this is not a JSR issue. In any case, some of the issues from other exploration and prototypes in this area: 1. Interaction with code on class path, esp. when you have two or more modules in the configuration for the boot layer that export the same package. 2. Split delegation issues that can arise when explicit modules on the module path do qualified exports to upgraded modules or even tool or automatic modules defined to the application class loader. 3. Visibility of types in non-exported packages, say where you have jdk.compiler defined to its own class loader but have code on the class path that makes use of types in conjunction with encapsulation busting options. 4. TCCL. There are other issues that arise from changing visibility but these are no different to issues that arise when using graphs of class loaders. -Alan. From reto.merz at abacus.ch Tue May 16 11:26:07 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Tue, 16 May 2017 13:26:07 +0200 Subject: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> Message-ID: > So I think Java Web Start is right to align with the policy for > applications launched on the command line. The difference is that in case of Webstart the Java version is controlled by the user. They are trained to install the latest Java version and keep it up-to-date. All our customers download Java via java.com They would not understand that they should stay with an old Java version until release X of software Y is available because they have never do that before. We use Webstart over 10 years and it is the first time we see that Oracle will break compatibility between Java N and N+1 and Oracle is already aware about it. The issue is easy to fix because only the Webstart launcher would be affected by the changes. This changes would give Webstart app devs more time to migrate. I think it is wrong to compare Webstart with a classic application and Oracle should add all SE/EE modules by default and also enable --permit-illegal-access by default in case ob Webstart. Regards Reto > -----Urspr?ngliche Nachricht----- > Von: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Gesendet: Dienstag, 16. Mai 2017 12:15 > An: Reto Merz; jigsaw-dev at openjdk.java.net > Betreff: Re: Java 9 EA 169 Webstart NoClassDefFoundError: > javax/xml/ws/Service > > On 16/05/2017 09:49, Reto Merz wrote: > > > Thanks Alan. > > > > Wouldn't it make sense to add all modules by default in Java 9 in case of > Webstart and print > > a warning to the Webstart console the first time a future non-default > module is accessed > > with the message that the JNLP must be extended with a --add- > modules=moduleXY in Java 10? > > > > So all the Webstart app devs would get more time to adjust the JNLP to > support Java9/10. > > > The proposal in JSR 379 is java.xml.ws and the other modules that > overlap with EE be deprecated (forRemoval=true) so that we can drop them > from Java SE in a future release (hopefully 10 if the JSR for Java SE 10 > agrees). > > So I think Java Web Start is right to align with the policy for > applications launched on the command line. > > -Alan. From Alan.Bateman at oracle.com Tue May 16 12:15:17 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 13:15:17 +0100 Subject: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> Message-ID: <9ea920d7-f52f-c596-e8f2-349569bc57a1@oracle.com> On 16/05/2017 12:26, Reto Merz wrote: >> So I think Java Web Start is right to align with the policy for >> applications launched on the command line. > The difference is that in case of Webstart the Java version is controlled by the user. > They are trained to install the latest Java version and keep it up-to-date. > All our customers download Java via java.com > They would not understand that they should stay with an old Java version until release X of software Y is available because they have never do that before. > We use Webstart over 10 years and it is the first time we see that Oracle will break compatibility between Java N and N+1 and Oracle is already aware about it. > The issue is easy to fix because only the Webstart launcher would be affected by the changes. > This changes would give Webstart app devs more time to migrate. > > I think it is wrong to compare Webstart with a classic application > and Oracle should add all SE/EE modules by default and also enable --permit-illegal-access > by default in case ob Webstart. > The JNLP fragment in my first mail can be used to resolve java.xml.ws for JNLP applications that depend on the JAX-WS version included in the JRE. It's predicated on the version of the runtime so you should be able to use the same JNLP for 6, 7, 8 and 9. I don't know when JDK 10 will be released and I also don't know for sure if JRE 10 will continue to bundle JAX-WS or not. Assuming it is removed then it will be a surprise as the JDK has never dropped APIs like this. I don't think we should feel too bad about that because each of these APIs are standalone technologies with their own download. BTW: We don't maintain these APIs in OpenJDK, instead they are maintained in upstream projects. I could imagine the migration guide for JDK 10 providing the JNLP need to download the standalone versions as libraries. As regards strong encapsulation then Java Web Start has support for the equivalent of `--add-exports` and `--add-opens`. I think we discussed this in a previous thread (you mentioned you inherited code that hacks String internals if I remember correctly). -Alan. From david.lloyd at redhat.com Tue May 16 13:15:06 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 16 May 2017 08:15:06 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <22f09f7a-59a0-0bbc-e4f7-f6ad9c04824c@redhat.com> On 05/16/2017 06:02 AM, Alan Bateman wrote: > On 12/05/2017 14:31, David M. Lloyd wrote: > >> : >> >>> There is a lot more to #5, something that will become clear when you >>> work through all the scenarios. The JSR and spec part are minor >>> though but I'd prefer to hold off until there is more discussion on >>> this topic in the JSR. >> >> I'd rather not hold off as the JSR essentially only has a couple of >> weeks left to live if there is not a revised PR. Could you please >> explain what you mean? Are you referring to jlink, jaotc, or >> something else? >> > For the most part, this is not a JSR issue. In any case, some of the > issues from other exploration and prototypes in this area: > > 1. Interaction with code on class path, esp. when you have two or more > modules in the configuration for the boot layer that export the same > package. I would say that isolated modules should be inaccessible from the class path, and can be specified accordingly. I don't think this would cause any particular usability concern or surprise; isolated modules are specifically intended to solve problems like package duplication that already means that the class path can't sensibly interact with them in an direct way. Also, if a user is going so far as to isolate modules from one another, I think it's a relatively small step to modularize whatever class path code they have. > 2. Split delegation issues that can arise when explicit modules on the > module path do qualified exports to upgraded modules or even tool or > automatic modules defined to the application class loader. I'll have a look at the upgraded modules code; I didn't realize that they would work differently from a normal module path module. Automatic modules should be subject to the same behavior as regular modules, but I'll double-check that as well. By tool modules, are you referring to jdk.compiler etc.? > 3. Visibility of types in non-exported packages, say where you have > jdk.compiler defined to its own class loader but have code on the class > path that makes use of types in conjunction with encapsulation busting > options. The current patch only allows application modules to be isolated. Otherwise this seems to be related to #1: I think it's reasonable to prevent class path code from accessing isolated modules. I think it's important to note that the class path does not need to have direct access to the full set of modular features - especially if such access would come at the cost of truncating that feature set. > 4. TCCL. For this one I was thinking that TCCL should be set to the class loader of the module that was executed. In the non-isolated case, this would be the application class loader, but in the isolated case it would depend on whether or not the main module was isolated. > There are other issues that arise from changing visibility but these are > no different to issues that arise when using graphs of class loaders. > > -Alan. > -- - DML From reto.merz at abacus.ch Tue May 16 13:52:10 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Tue, 16 May 2017 15:52:10 +0200 Subject: AW: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: <9ea920d7-f52f-c596-e8f2-349569bc57a1@oracle.com> References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> <9ea920d7-f52f-c596-e8f2-349569bc57a1@oracle.com> Message-ID: First, I want to say thanks for your help in this and the previous threads. > The JNLP fragment in my first mail can be used to resolve java.xml.ws > for JNLP applications that depend on the JAX-WS version included in the > JRE. It's predicated on the version of the runtime so you should be able > to use the same JNLP for 6, 7, 8 and 9. > As regards strong encapsulation then Java Web Start has support for the > equivalent of `--add-exports` and `--add-opens`. I think we discussed > this in a previous thread (you mentioned you inherited code that hacks > String internals if I remember correctly). Yes we have added tons of --add-exports and --add-opens to our JNLP and just also a "--add-modules=java.xml.ws". That works great and testing with Java 9 are going well so far. The --add-exports, -add-opens and --add-modules entries is a good list to see what we need to fix. Of course it is our target to remove this list but this take some time. Our problem is that our "old" JNLP, which works with Java 6, 7 and 8 does no longer work with Java 9. This could be easily solved by the Webstart launcher in Java 9. Java 9 should just generate warnings in the Webstart Java console to inform which --add-exports/-add-opens or --add-modules is missing in the JNLP and that this JNLP will not work with Java 10. This will give us, and especially our customers, more time to migrate. To explain that a bit more: Our JNLP is not a static resource, it is generated server-side. So our customers need to update the server installations which is a big and costly task in our business field (ERP software). We have customers which hosts server installations which is used by hundreds and thousands of users in their daily work. Such big customers can not quickly install a new software update we have just released. They need weeks to plan and test an upcoming server-side software update before they can install it on the production system. Java 9 will be released on 2017/07/27, so we are under high business pressur. Furthermore we have no guarantee that the "new" JNLP, which works with 9-ea+169, will finally works with the final Java 9 release and none can guarantee that until the final Java 9 version is really released. This is also the reason we have improved the JNLP stuff: In further releases of our software, we allow to modify/change the server-side JNLP generation in a more flexible way via config files to defuse JNLP related issues. > I don't know when JDK 10 will be released and I also don't know for sure > if JRE 10 will continue to bundle JAX-WS or not. Assuming it is removed > then it will be a surprise as the JDK has never dropped APIs like this. > I don't think we should feel too bad about that because each of these > APIs are standalone technologies with their own download. BTW: We don't > maintain these APIs in OpenJDK, instead they are maintained in upstream > projects. I could imagine the migration guide for JDK 10 providing the > JNLP need to download the standalone versions as libraries. Yes and it is the right way to deprecate those modules in Java 9, as you have done. So we aware of the deprecation and we have time to prepare a solution for this. This is what is going wrong with Java 9 and the JNLP: Java 8 has give us no hint that this JNLP will no longer work with Java 9 and what we should do. Otherwise it would be enough time to prepare our software. The Webstart in Java 9 EA has started working just a few weeks/months ago (around EA 15x). So there was also no chance to test and prepare our software with Java 9 EA. That's why I propose to enable all modules and set --permit-illegal-access by default in case of the Webstart for (and only for) Java 9. BTW: I would also welcome if the Oracle Webstart would have a specification and been part of the OpenJDK or somewhere as open source project. Regards Reto From mandy.chung at oracle.com Tue May 16 16:00:38 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 May 2017 09:00:38 -0700 Subject: jdeps --generate-open-module In-Reply-To: <537D2161-CEF3-4802-A14A-9969E64A21F1@luminis.eu> References: <537D2161-CEF3-4802-A14A-9969E64A21F1@luminis.eu> Message-ID: <68E634FB-E968-4F3E-81F2-63A5FC0BDC2E@oracle.com> > On May 15, 2017, at 12:43 PM, Sander Mak wrote: > > When executing `jdeps --generate-module-info ./out `, a module descriptor exporting all packages is created. When using `--generate-open-module`, a module descriptor with an open module is generated, but no packages are exported. I would expect to be able to generate a module descriptor that is as close to the automatic module behavior of the JAR in question, as starting point for modularization. There seems to be no way to generate an open module exporting all packages, which is the closest to automatic module behavior as possible. What's the intended use-case of `--generate-open-module` as it stands? jdeps --generate-module-info generates a normal module exporting all packages. To make it an open module (that exports all packages), you can use this option and then add ?open? keyword in the generated module-info. They may be cases for an open module with no exports for example for frameworks to access reflectively. jdeps -?generate-open-module is the option to generate such module-info.java. Of course it?s easy for jdeps to generates `exports` for all packages in this case. This gives the author the chance to consider whether a module should open all packages for reflective access as well as exporting all packages. Mandy From greggwon at cox.net Tue May 16 18:11:11 2017 From: greggwon at cox.net (Gregg Wonderly) Date: Tue, 16 May 2017 13:11:11 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: At some level, this is the problem that is paramount on the release of JDK-9. Earlier Mark asked if the Eclipse foundation had to approve or be ready to support all of what JDK-9/Jigsaw supports before it could be released. The statement below seems to stipulate that ?all Java software must be convertible to what JDK-9 is demanding? or it?s no longer going to be usable with JDK-9 and later. That is not a backward compatible premise for evolution of Java. So ultimately, what is being demanded, is that the entire world of Java should be continuously reading this group, responding to releases, and making sure that their software systems are compatible so that all users of all exist Java applications will be ready to be replaced by working version once JDK-9 is released. The question of is JDK-9 compatible software also able to run on JDK-8 as well as the fact that JDK-8 compatible software may largely be unable to run on JDK-9 is still something that seems to get shrugs of indifference from the Jigsaw team. Evolving Java to have better modularity is a great idea. But somehow we still have statements about ?Jigsaw modularity includes isolationist based security, deal with it" is still the primary concept around most replies to questions on the list about how moving forward with existing applications/libraries and standard development practices. It seems strange, that with such a wide sweeping change to how Java works, that the JigSaw team still seems amazingly confused/frustrated that people are troubled by all the extra work that they will have to do, to somehow untangle their software systems from other software systems which Jigsaw has seem to put on the hook for using the features of Java prior to JDK-8. If we really cannot actually keep from breaking 90% of existing Java in the market place when this new JDK release goes out, how valuable is JigSaw really? How much community focus does this suggest there is in the design and implementation? How many people/teams/companies are going to look at this as some form of decline of what Java, write once, run anywhere, actually means to them and their development team? I still come back around to the SecurityManager being a much better focus point for denial of access. It is just really odd that because you want to do this at the JRE implementation level, and want to do control access for all software which has the same declarative exposure, that we all have no choice at all, for how we will be exposed to the problems that will be created when auto updates install JDK-9 over the top of JDK-8. Gregg > On May 16, 2017, at 5:39 AM, Alan Bateman wrote: > > On 12/05/2017 14:31, David M. Lloyd wrote: > >> : >> >>> #4 seems to be working around the outcome of issue #CyclicDependences in the JSR. I also don't wish to comment on that except to say that introducing system properties to skip specified checks is highly problematic from a conformance perspective. >> >> Can you explain what you mean by that? I'm more than happy to convert these into -X type arguments to the runtime (or, of course, to simply make this the standard behavior), but I thought this would be a simpler and safer approach (at least for a first pass). >> > Disabling specified behavior creates a conformance issue. It doesn't matter if it's a CLI option or a system property. > > In any case, proposing a patch here is not the right way to re-open JSR issue #CyclicDependences. Instead, I think it would be better to work through a number of migration scenarios and see what real-issues you run into. Do you run into cycles that can't be addressed with clean-up and migrating to services for examples? > > -Alan. From Alan.Bateman at oracle.com Tue May 16 20:01:06 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 May 2017 21:01:06 +0100 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <1a09b70e-6570-c78f-032a-4bbe6132dafd@oracle.com> On 16/05/2017 19:11, Gregg Wonderly wrote: > At some level, this is the problem that is paramount on the release of JDK-9. Earlier Mark asked if the Eclipse foundation had to approve or be ready to support all of what JDK-9/Jigsaw supports before it could be released. > > The statement below seems to stipulate that ?all Java software must be convertible to what JDK-9 is demanding? or it?s no longer going to be usable with JDK-9 and later. That is not a backward compatible premise for evolution of Java. So ultimately, what is being demanded, is that the entire world of Java should be continuously reading this group, responding to releases, and making sure that their software systems are compatible so that all users of all exist Java applications will be ready to be replaced by working version once JDK-9 is released. The question of is JDK-9 compatible software also able to run on JDK-8 as well as the fact that JDK-8 compatible software may largely be unable to run on JDK-9 is still something that seems to get shrugs of indifference from the Jigsaw team. > > Evolving Java to have better modularity is a great idea. But somehow we still have statements about ?Jigsaw modularity includes isolationist based security, deal with it" is still the primary concept around most replies to questions on the list about how moving forward with existing applications/libraries and standard development practices. > > It seems strange, that with such a wide sweeping change to how Java works, that the JigSaw team still seems amazingly confused/frustrated that people are troubled by all the extra work that they will have to do, to somehow untangle their software systems from other software systems which Jigsaw has seem to put on the hook for using the features of Java prior to JDK-8. > > If we really cannot actually keep from breaking 90% of existing Java in the market place when this new JDK release goes out, how valuable is JigSaw really? How much community focus does this suggest there is in the design and implementation? How many people/teams/companies are going to look at this as some form of decline of what Java, write once, run anywhere, actually means to them and their development team? > > I still come back around to the SecurityManager being a much better focus point for denial of access. It is just really odd that because you want to do this at the JRE implementation level, and want to do control access for all software which has the same declarative exposure, that we all have no choice at all, for how we will be exposed to the problems that will be created when auto updates install JDK-9 over the top of JDK-8. > This thread is not about strong encapsulation. It's also not a thread about a JDK 9 compatibility issue. It is instead about optional migration to modules where you address technical debt and refactor to use services as part of making your code modular. -Alan. From adinn at redhat.com Wed May 17 08:08:08 2017 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 17 May 2017 09:08:08 +0100 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: On 16/05/17 19:11, Gregg Wonderly wrote: > If we really cannot actually keep from breaking 90% of existing Java > in the market place when this new JDK release goes out, how valuable > is JigSaw really? citation needed? regards, Andrew Dinn ----------- From michael.rasmussen at zeroturnaround.com Wed May 17 08:12:40 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Wed, 17 May 2017 11:12:40 +0300 Subject: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> <9ea920d7-f52f-c596-e8f2-349569bc57a1@oracle.com> Message-ID: On 16 May 2017 at 16:52, Reto Merz wrote: > They need weeks to plan and test an upcoming server-side > software update before they can install it on the production system. > Java 9 will be released on 2017/07/27, so we are under high business pressur. Assuming Java 9 follows a similar path at Java 8, it will be around 6 months after release, before Java 9 is the default download on java.com, and another 6 months before the auto-updater will update JRE installations from java 8 to java 9. Yes, for those users who actively seek out java 9 on day one, there can be issues, but for the average consumer, there is a 6-12 month period from release before Java 9 will be on their computers. Yes, we still have to go through all the work of migrating to Java 9, but should the above hold true for Java 9, we do at least have a bit of breathing room, before it is actively being pushed to users. /Michael From javadevlist at javasoft.de Wed May 17 09:54:07 2017 From: javadevlist at javasoft.de (wzberger) Date: Wed, 17 May 2017 11:54:07 +0200 Subject: How to load resources from base module Message-ID: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> Looks like I'm having an encapsulation issue with resources. Class B belongs to module b and extends Class A in module a. Class B provides resources which should be loaded from base class A. However, calling getInstance().getClass().getResource(...) from a static method in class A fails while calling from class B works fine. Because getInstance() returns Class B I wonder why the resource can't be loaded. Any recommendation on how to get this working? Thanks, Wolfgang From Alan.Bateman at oracle.com Wed May 17 10:23:17 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 May 2017 11:23:17 +0100 Subject: How to load resources from base module In-Reply-To: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> Message-ID: On 17/05/2017 10:54, wzberger wrote: > Looks like I'm having an encapsulation issue with resources. Class B > belongs to module b and extends Class A in module a. Class B provides > resources which should be loaded from base class A. > > However, calling > > getInstance().getClass().getResource(...) > > from a static method in class A fails while calling from class B works > fine. Because getInstance() returns Class B I wonder why the resource > can't be loaded. Any recommendation on how to get this working? The details are in the javadoc but in summary: Code in `b` can locate any resource in `b`. Code in `a` can any locate any ".class" resource in `b`. It can also locate any resources in "non-package" locations such as META-INF/**. Resources that are located in b's packages are encapsulated and so `b` needs to `opens to a` to allow code in module `a` to locate the resources. -Alan. From javadevlist at javasoft.de Wed May 17 17:22:41 2017 From: javadevlist at javasoft.de (wzberger) Date: Wed, 17 May 2017 19:22:41 +0200 Subject: How to load resources from base module In-Reply-To: References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> Message-ID: <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> The resources are mainly images and XML files, located in separate packages. By adding the opens clause it works fine for Class B (called from module a) - however, it does not work for Class A (returns null). I wonder if this is the intended behavior? -Wolfgang > >> Looks like I'm having an encapsulation issue with resources. Class B >> belongs to module b and extends Class A in module a. Class B provides >> resources which should be loaded from base class A. >> >> However, calling >> >> getInstance().getClass().getResource(...) >> >> from a static method in class A fails while calling from class B >> works fine. Because getInstance() returns Class B I wonder why the >> resource can't be loaded. Any recommendation on how to get this working? > The details are in the javadoc but in summary: > > Code in `b` can locate any resource in `b`. > > Code in `a` can any locate any ".class" resource in `b`. It can also > locate any resources in "non-package" locations such as META-INF/**. > Resources that are located in b's packages are encapsulated and so > `b` needs to `opens to a` to allow code in module `a` to > locate the resources. > > -Alan. > > > > From eric at tibco.com Wed May 17 17:26:22 2017 From: eric at tibco.com (Eric Johnson) Date: Wed, 17 May 2017 10:26:22 -0700 Subject: Evidence of incompatibility (was Some suggested patches and improvements) Message-ID: On Wed, May 17, 2017 at 1:08 AM, Andrew Dinn wrote: > On 16/05/17 19:11, Gregg Wonderly wrote: > > > > > If we really cannot actually keep from breaking 90% of existing Java > > in the market place when this new JDK release goes out, how valuable > > is JigSaw really? > > citation needed? > I mostly ignore jigsaw, and check in every now and then. I have a few co-workers that have poked at migrating their products to Java 9. So far as I know, nobody has succeeded yet. With significant regularity, I see issues pop up on this list that have odd problems, or persist in being unresolved. One of my favorites at the moment is automatic module names - a problem that Jigsaw caused for itself. Maybe that one is resolved for now, but I'm pretty certain that questions will come flooding back once Java 9 GAs. As near as I can tell, applications that compile and run under Java 8 will mostly *not* "just work" with Java 9 JRE. And that seems to be the lived experience of my co-workers. If a project is lucky, the only changes necessary will involve command line parameters. If a team is unlucky, they will need to rebuild for Java 9. If a team is really unlucky, they will need to partially or fully modularize. At which point some even more juggling is required to continue to support Java 7 & 8, if that's required by customers. My overall concerns for Jigsaw: https://medium.com/@one.eric.johnson/java-9-jigsaw-troubles-4fc406ef41e0 I'm not sure what citations you expect to see. There's probably nobody out there who can afford to pre-flight an EA build of Java 9 against all their products to see what the actual costs are going to be. Based on anecdotal evidence from this mailing list, significant players in the Java ecosystem - build tools, IDEs, critical libraries - have all had to fix unexpected breakages with Java 9. Obviously, the ones that don't break don't typically show up, so this is a self-selecting example, but an important one. However, even something as simple as requiring changes to command line parameters in order to launch a program compiled for Java 8 is a breaking change. The Jigsaw team seems to be taking this as a mere complaint, rather than as a genuine compatibility issue. Here's a challenge back to the Jigsaw team. Can I still do java -jar ... every existing Java application (without recompile!) that currently launches that way? I'm even willing to cut some slack and ignore applications that use com.sun APIs that have been "private" for years. Will that still work? The Jigsaw community should be able to provide evidence that's still possible, not that we should be required to provide evidence that it isn't. Eric. > regards, > > > Andrew Dinn > ----------- > From Alan.Bateman at oracle.com Wed May 17 17:52:47 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 May 2017 18:52:47 +0100 Subject: How to load resources from base module In-Reply-To: <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> Message-ID: <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> On 17/05/2017 18:22, wzberger wrote: > The resources are mainly images and XML files, located in separate > packages. By adding the opens clause it works fine for Class B (called > from module a) - however, it does not work for Class A (returns null). > I wonder if this is the intended behavior? I'm not aware of any bugs in this area. Can you expand a bit more on "returns null" case. If the packages are open as you say then code in both `a` and `b` should be able to locate the resources in `b`. -Alan From Alan.Bateman at oracle.com Wed May 17 18:50:07 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 May 2017 19:50:07 +0100 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: References: Message-ID: <8c693b74-628c-7749-9c52-cb2ad4c0f3eb@oracle.com> On 17/05/2017 18:26, Eric Johnson wrote: > : > Here's a challenge back to the Jigsaw team. Can I still do java -jar ... > every existing Java application (without recompile!) that currently > launches that way? I'm even willing to cut some slack and ignore > applications that use com.sun APIs that have been "private" for years. Will > that still work? The Jigsaw community should be able to provide evidence > that's still possible, not that we should be required to provide evidence > that it isn't. FWIW, the JDK has never been able to guarantee this at major releases. A typical JDK major release will have ~20000 changes that translate into a lot of code so there is a lot of potential for breakage or subtle behavior or performance changes. Even mundane things like rev'ing the class file version at every major release can be disruptive. Nothing to do with Project Jigsaw but there are many one-off changes in JDK 9 that will make it more disruptive than previous releases, e.g. moving to G1 as the default collector, switching to use CDLR locale data by default, the new version-string scheme, changes to the layout of the run-time image, .... it's a long list that will eventually show up as extensive release and compatibility notes. As regards executable JARs then please bring up any specific issues if you run into them. The reason that there are weekly EA builds to get as much testing and feedback as possible. If you search around then you'll see that all the IDEs have been working with the JDK 9 builds, same thing for many other important tools and libraries in the eco system. We know of course that only a small percentage of developers will have the time or interest but having at least some people trying out builds and reporting issues (to the right place) is really important and has been part of how major releases have been developed for a long time. As regards your command on cutting slack then be aware that the issue of existing applications run with `java -jar` but relying on direct access to the JDK internals has been discussed many times. There are JAR file attributes that can be added to the main manifest that are equivalent to the encapsulation busting command line options. -Alan. From robbiexgibson at yahoo.com Wed May 17 19:56:15 2017 From: robbiexgibson at yahoo.com (Robert Gibson) Date: Wed, 17 May 2017 21:56:15 +0200 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: <8c693b74-628c-7749-9c52-cb2ad4c0f3eb@oracle.com> References: <8c693b74-628c-7749-9c52-cb2ad4c0f3eb@oracle.com> Message-ID: On 17 May 2017, at 20:50, Alan Bateman wrote: > > On 17/05/2017 18:26, Eric Johnson wrote: > >> : >> Here's a challenge back to the Jigsaw team. Can I still do java -jar ... >> every existing Java application (without recompile!) that currently >> launches that way? I'm even willing to cut some slack and ignore >> applications that use com.sun APIs that have been "private" for years. Will >> that still work? The Jigsaw community should be able to provide evidence >> that's still possible, not that we should be required to provide evidence >> that it isn't. > FWIW, the JDK has never been able to guarantee this at major releases. A typical JDK major release will have ~20000 changes that translate into a lot of code so there is a lot of potential for breakage or subtle behavior or performance changes. Even mundane things like rev'ing the class file version at every major release can be disruptive. Nothing to do with Project Jigsaw but there are many one-off changes in JDK 9 that will make it more disruptive than previous releases, e.g. moving to G1 as the default collector, switching to use CDLR locale data by default, the new version-string scheme, changes to the layout of the run-time image, .... it's a long list that will eventually show up as extensive release and compatibility notes. > And of course the intersection of these issues makes things even more complicated. At this point I would settle for being able to add a command-line flag to get our app to run, but despite testing EA builds for over a year and assiduously flagging issues as we came across them it?s still not possible for us (intersection of CLDR change and Webstart) https://bugs.openjdk.java.net/browse/JDK-8160828 Low-energy rant ends here in resignation, Robert From Alan.Bateman at oracle.com Wed May 17 20:30:54 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 May 2017 21:30:54 +0100 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: References: <8c693b74-628c-7749-9c52-cb2ad4c0f3eb@oracle.com> Message-ID: <9174aca5-f111-8aae-293e-202b413f6af1@oracle.com> On 17/05/2017 20:56, Robert Gibson wrote: > : > And of course the intersection of these issues makes things even more complicated. At this point I would settle for being able to add a command-line flag to get our app to run, but despite testing EA builds for over a year and assiduously flagging issues as we came across them it?s still not possible for us (intersection of CLDR change and Webstart) > https://bugs.openjdk.java.net/browse/JDK-8160828 > This isn't Jigsaw or modules or course but just to say that the transition to CLDR locale data has been a multi-release effort (details in JEP 127 and 252). The CDLR support went into JDK 8 with a system property to enable. The default changed in JDK 9 but the system property can be used to go back to the legacy JRE locale data. I think what you saying is that we failed to white list that property for JNLP applications. This seems like something that should be fixed. -Alan From forax at univ-mlv.fr Wed May 17 22:34:41 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 18 May 2017 00:34:41 +0200 (CEST) Subject: An alternative to "restricted keywords" In-Reply-To: <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> Message-ID: <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> I want to answer this before we start the meetings because i really think that restricted keyword as i propose solve the issues Stephan raised. ----- Mail original ----- > De: "Stephan Herrmann" > ?: jigsaw-dev at openjdk.java.net > Envoy?: Mardi 16 Mai 2017 11:49:45 > Objet: Re: An alternative to "restricted keywords" > Thanks, Remi, for taking this to the EG list. > > Some collected responses: > > > Remi: "from the user point of view, '^' looks like a hack" > > This is, of course, a subjective statement. I don't share this view > and in years of experience with Xtext-languages (where this concept > is used by default) I never heard any user complain about this. > > More importantly, I hold that such aesthetic considerations are of > much lesser significance than the question, whether we can explain > - unambiguously explain - the concept in a few simple sentences. > Explaining must be possible at two levels: in a rigorous specification > and in simple words for users of the language. I'm not against ^, or ` as it has already asked to escape an identifier, but as you said it's a pervasive change that applies on the whole grammar while i think that with restricted keyword (that really should be called local keywords) the changes only impact the grammar that specifies a module-info.java > > Remi: "a keyword which is activated if you are at a position in the > grammar where it can be recognized". > > I don't think 'being at a position in the grammar' is a good way of > explaining. Parsing doesn't generally have one position in a grammar, > multiple productions can be active in the same parser state. > Also speaking of a "loop" for modifiers seems to complicate matters > more than necessary. > > Under these considerations I still see '^' as the clearest of all > solutions. Clear as a specification, simple to explain to users. Eclipse uses a LR parser, for a LR parser, position == dotted production as i have written earlier, so no problem because it corresponds to only one parser state. Note that even if one do not use an LR or a LL parser, most hand written parser i've seen, javac is one of them, also refers to dotted production in the comments of the corresponding methods. > > > > Peter spoke about module names vs. package names. > > I think we agree, that module names cannot use "module words", > whereas package names should be expected to contain them. yes, that the main issue, package names may contains unqualified name like 'transitive, ''with' or 'to'. but i think people will also want to use existing package or more exactly prefix of existing package as module name, so we should also support having restricted keyword name as part of a module name. The grammar is: open? module module_name { requires (transitive | static)* module_name; exports package_name; exports package_name to module_name1, module_name2; opens package_name; opens package_name to module_name1, module_name2; uses xxx; provides xxx with xxx, yyy; } If we just consider package name, only 'opens' and 'exports' are followed by a package name and a package name can only been followed by ';' or 'to', so once 'opens' is parsed, you know that you can have only an identifier so if it's not an identifier by one of the restricted keywords, it should be considered as an identifier. As i said earlier, the scanner can see the restricted keyword as keyword and before feeding the token to the parser, you can check the parser state to see if the keyword as to be lowered to an identifier or not. For module name, there is the supplementary problem of transitive, because if a module starts with transitive, you can have a conflict. As i said earlier, instead of using the next token to know if transitive is the keyword or part of the module name, i think we should consider it as a keyword, as the JLS said a restricted keyword is activated when it can appear, so "requires transitive" is not a valid directive. > > Remi: "you should use reverse DNS naming for package so no problem :)" > > "to" is a "module word" and a TLD. > I think we should be very careful in judging that a existing conflict > is not a real problem. Better to clearly and rigorously avoid the > conflict in the first place. to as the first part of a package/module and to as in exports ... to can not be present on the same dotted production, because exports as to be followed by a package_name so 'to' here means the start of a package name and then because a package name can not ends with '.' you always know if you are inside the production recognizing the package_name or outside matching the to of the directive exports. > > > > Some additional notes from my side: > > In the escape-approach, it may be prudent to technically allow > escaping even words that are identifiers in Java 9, but could become > keywords in a future version. This ensures that modules which need > more escaping in Java 9+X can still be parsed in Java 9. yes, that's why i think that escaping is not the right mechanism here, because we want to solve a very local problem so we do not need a global grammar-wise way to solve our problem. > > > Current focus was on names of modules, packages and types. > A complete solution must also give an answer for annotations on modules. > Some possible solutions: > a. Assume that annotations for modules are designed with modules in mind > and thus have to avoid any module words in their names. > b. Support escaping also in annotations > c. Refine the scope where "module words" are keywords, let it start only > when the word "module" or the group "open module" has been consumed. > This would make the words "module" and "open" special, as being > switch words, where we switch from one language to another. > (For this I previously coined the term "scoped keywords" [1]) For annotation, again, because annotation name are qualified, you now when you see 'module' if you are in the middle of the annotation name or if you are outside. > > > I think we all agree that the conflicts we are solving here are rare > corner cases. Most names do not contain module words. Still, from a > conceptual and technical p.o.v. the solution must be bullet proof. > But there's no need to be afraid of module declarations being spammed > with dozens of '^' characters. Realistically, this will not happen. > I agree, and i strongly believe that scoped keyword, local keywords or restricted keywords, i.e. whatever the name, keywords that are keywords or identifiers depending on the parser state are the general mechanism that solve our problem. > Stephan > > [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 > R?mi From greggwon at cox.net Thu May 18 00:29:34 2017 From: greggwon at cox.net (Gregg Wonderly) Date: Wed, 17 May 2017 19:29:34 -0500 Subject: Some suggested patches and improvements In-Reply-To: References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> Message-ID: <1E02E868-9599-43CC-A099-9853FBE2C5F7@cox.net> > On May 17, 2017, at 3:08 AM, Andrew Dinn wrote: > > On 16/05/17 19:11, Gregg Wonderly wrote: > > > >> If we really cannot actually keep from breaking 90% of existing Java >> in the market place when this new JDK release goes out, how valuable >> is JigSaw really? > > citation needed? This is going to seem like more ranting, but alas, I am mentally worn out by Jigsaw and the devolution of Java that it reinforces by continuing to only worry about a limited set of Server application environments with the Jigsaw team apparently, completely ignorant of the vast application environments where Java has been deployed, including clickable jar files. I understand that you might feel this is an excessive rant. But, practically I know of Java applications around the world which all use things like Spring. I know of many others which are using reflection and all kinds of .setAccessible() behaviors to manage ?serialization? and other tasks for foreign classes. As someone who doesn?t use Java-EE or any other ?well used? platform, but rather uses many things I created myself well before those things existed, I suspect that I will find breakage in my larger apps simply because of setAccessible() which I use for introspection in tools (via Introspector etc.) and other things (like serialization). Maven seems to be something that the Jigsaw team had no real knowledge about. It just suddenly introduced a flurry of conversation on the list as something that was unimportant for modularization and jar reference issues. Another pointer to the lack of foresight the Jigsaw team seemed to have into the realities of how Java is actually used around the world. I have not tried JDK-9 on any of my applications primarily because I have zero spare work cycles to do that at this point. This list content and the general visible attitude of the team towards backwards compatibility tell me that 1) I will have problems, 2) I could mention them here, but little will happen to make my investigation create less work in the future for me. So, I consider that JDK-9 GA will in fact be a platform departure moment for me. I will be looking at Swift and/or Go and just moving on. My experience with Java started out great. I had a lot of good experiences at JavaOne, and enjoyed getting to be a part of the Sun developers advisory council for the first few years. Then I learned that there were about 3 people at Sun who only cared about Java as a platform for selling more Sun servers, and the recommendations of the SDAC, to Sun seemed to fall on deaf ears. James Gosling told us he could feel our pain and didn?t really know how to help us, help Sun make better choices. So, now here we are into the Oracle legacy. Two of the same 3 people seem to still be around (Jonathan fired one a while back at Sun) making huge influence and still feeling like a wreaking ball, destroying much opportunity for Java to actually unify the software development world. If it wasn?t clear enough based on the continued development of more (procedural, object oriented) programming languages around the world, Java is way off course to ever be a general solution to software development. The server environment is like 2% of the space Java could succeed in. The Java Card market is a fixed space which should of evolved to include things like what Apple Pay does. Yet chip cards in the US are floundering because the technology is so slow to boot and run. Now, credit card transactions are 4-5x slower than they used to be. JavaFX wasn?t much of a concern here on the list for some time. There was no conversation about anything regarding compatibility. It?s supposed to be a big part of the evolution of the desktop environment, and you?d think that Oracle employees would be working together, across the company. It?s that kind of neglect of the ?impact? on the platform which makes me feel like everything about JigSaw is a closed room idea which got a lot of work by a small group, and now with the level of investment made, anything that?s voiced as negative about what?s been done, is just viewed as a whinny rant instead of as something valuable for the team to learn about. There are literally, I would bet, 10,000 more things to learn about Jigsaw breaking/impacting that have zero value to the community, but will make the Jigsaw team feel strong and empowered because they are ?making evolution of the platform possible?. At the beginning it was mostly those 3 people?Now I am not sure that its worth worrying about who is responsible as much as it will be to figure out the fastest way out? I still marvel at the sheer magnitude of rework that Jigsaw is imposing on the world at large, and what small, literally unimportant impact that it will have after that rework occurs. Gregg From rgoers at apache.org Thu May 18 03:28:47 2017 From: rgoers at apache.org (Ralph Goers) Date: Wed, 17 May 2017 20:28:47 -0700 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: References: Message-ID: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> I am afraid I have to echo these sentiments to some degree. In trying to get Log4j to support Java 9 I first tried to use a multi-release jar. This failed miserably when the OSGi build tool failed over finding java classes under META-INF. Then it proceeded to complain about the module-info.java files. Why these are java syntax instead of json or something more sensible for something that only contains declarations is a mystery to me. FWIW - the OSGi people don?t seem interested in supporting these new features - https://issues.apache.org/jira/browse/FELIX-5592 . I have been able to work around some of these issues but it has made the Log4j build very fragile and I haven?t really begun to see what happens when log4j is actually modularized or runs in an application that is. Ralph > On May 17, 2017, at 10:26 AM, Eric Johnson wrote: > > On Wed, May 17, 2017 at 1:08 AM, Andrew Dinn wrote: > >> On 16/05/17 19:11, Gregg Wonderly wrote: >> >> >> >>> If we really cannot actually keep from breaking 90% of existing Java >>> in the market place when this new JDK release goes out, how valuable >>> is JigSaw really? >> >> citation needed? >> > > I mostly ignore jigsaw, and check in every now and then. > > I have a few co-workers that have poked at migrating their products to Java > 9. So far as I know, nobody has succeeded yet. > > With significant regularity, I see issues pop up on this list that have odd > problems, or persist in being unresolved. One of my favorites at the moment > is automatic module names - a problem that Jigsaw caused for itself. Maybe > that one is resolved for now, but I'm pretty certain that questions will > come flooding back once Java 9 GAs. > > As near as I can tell, applications that compile and run under Java 8 will > mostly *not* "just work" with Java 9 JRE. And that seems to be the lived > experience of my co-workers. If a project is lucky, the only changes > necessary will involve command line parameters. If a team is unlucky, they > will need to rebuild for Java 9. If a team is really unlucky, they will > need to partially or fully modularize. At which point some even more > juggling is required to continue to support Java 7 & 8, if that's required > by customers. > > My overall concerns for Jigsaw: > https://medium.com/@one.eric.johnson/java-9-jigsaw-troubles-4fc406ef41e0 > > I'm not sure what citations you expect to see. There's probably nobody out > there who can afford to pre-flight an EA build of Java 9 against all their > products to see what the actual costs are going to be. Based on anecdotal > evidence from this mailing list, significant players in the Java ecosystem > - build tools, IDEs, critical libraries - have all had to fix unexpected > breakages with Java 9. Obviously, the ones that don't break don't typically > show up, so this is a self-selecting example, but an important one. > > However, even something as simple as requiring changes to command line > parameters in order to launch a program compiled for Java 8 is a breaking > change. The Jigsaw team seems to be taking this as a mere complaint, rather > than as a genuine compatibility issue. > > Here's a challenge back to the Jigsaw team. Can I still do java -jar ... > every existing Java application (without recompile!) that currently > launches that way? I'm even willing to cut some slack and ignore > applications that use com.sun APIs that have been "private" for years. Will > that still work? The Jigsaw community should be able to provide evidence > that's still possible, not that we should be required to provide evidence > that it isn't. > > Eric. > > >> regards, >> >> >> Andrew Dinn >> ----------- >> > From Alan.Bateman at oracle.com Thu May 18 06:28:01 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 07:28:01 +0100 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> References: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> Message-ID: <08166b2c-ab1a-4a93-076b-47ed4dcd61df@oracle.com> On 18/05/2017 04:28, Ralph Goers wrote: > I am afraid I have to echo these sentiments to some degree. In trying to get Log4j to support Java 9 I first tried to use a multi-release jar. This failed miserably when the OSGi build tool failed over finding java classes under META-INF. Then it proceeded to complain about the module-info.java files. Why these are java syntax instead of json or something more sensible for something that only contains declarations is a mystery to me. FWIW - the OSGi people don?t seem interested in supporting these new features - https://issues.apache.org/jira/browse/FELIX-5592 . > If I read this correctly then you'll looking to use MR JARs so that you can use the new stack walking API when on JDK 9 or later. This is exactly the type of use-case that MR JARs are intended for. For the JIRA issue then it might be worth including a link to JEP 238 and also the new JarFile constructor which makes it easy and transparent that the resources are located in versioned sectioned of the MR JAR. The Maven maintainers on this list are very familiar with MR JARs. I suspect it might take time for support to trickle into some of the plugins that need to know about them. -Alan. From Alan.Bateman at oracle.com Thu May 18 07:06:30 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 08:06:30 +0100 Subject: Some suggested patches and improvements In-Reply-To: <1E02E868-9599-43CC-A099-9853FBE2C5F7@cox.net> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> <1E02E868-9599-43CC-A099-9853FBE2C5F7@cox.net> Message-ID: <5bfc2f6a-7095-3372-4760-1cb2ff870035@oracle.com> On 18/05/2017 01:29, Gregg Wonderly wrote: > > I understand that you might feel this is an excessive rant. But, practically I know of Java applications around the world which all use things like Spring. Spring seems to be making great progress, here's a recent write-up from Juergen Hoeller: https://spring.io/blog/2017/05/08/spring-framework-5-0-goes-rc1 > I know of many others which are using reflection and all kinds of .setAccessible() behaviors to manage ?serialization? and other tasks for foreign classes. As someone who doesn?t use Java-EE or any other ?well used? platform, but rather uses many things I created myself well before those things existed, I suspect that I will find breakage in my larger apps simply because of setAccessible() which I use for introspection in tools (via Introspector etc.) and other things (like serialization). Java serialization works as before. Custom serialization libraries is a challenge of course as it is fundamentally incompatible with encapsulation. This is why sun.reflect.ReflectionFactory has not been encapsulation (see JEP 260) and furthermore extended to allow custom serialization libraries work with a modular platform. That support went into a JDK 8 update too. > > Maven seems to be something that the Jigsaw team had no real knowledge about. It just suddenly introduced a flurry of conversation on the list as something that was unimportant for modularization and jar reference issues. Another pointer to the lack of foresight the Jigsaw team seemed to have into the realities of how Java is actually used around the world. We have been working with the Maven maintainers for a long time (since 2015). Here's the wiki page that Robert Scholte, Herv? Boutemy, and others have been using the track status: https://cwiki.apache.org/confluence/display/MAVEN/Java+9+-+Jigsaw I also note a recent tweet from Robert, I can only guess that he's been asked a lot about this too. https://twitter.com/rfscholte/status/864560567767498752 > > > > JavaFX wasn?t much of a concern here on the list for some time. There was no conversation about anything regarding compatibility. It?s supposed to be a big part of the evolution of the desktop environment, and you?d think that Oracle employees would be working together, across the company. It?s that kind of neglect of the ?impact? on the platform which makes me feel like everything about JigSaw is a closed room idea I think this is insulting to Kevin Rushforth and others in the OpenJFX project that we have been working with to ensure that JavaFX works great on JDK 9. The JDK downloads have included a fully modularized JavaFX for over year. -Alan From javadevlist at javasoft.de Thu May 18 07:16:28 2017 From: javadevlist at javasoft.de (wzberger) Date: Thu, 18 May 2017 09:16:28 +0200 Subject: How to load resources from base module In-Reply-To: <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> Message-ID: Here is a simple example which demonstrates the issue - still something missing? //------------ MODULE A module com.a { exports com.a; } package com.a; public class ClassA{ public ClassA() { //success System.out.println(getClass().getResource("/com/b/resources/test.txt")); //fail - null System.out.println(ClassA.class.getResource("/com/b/resources/test.txt")); } } //------------ MODULE B module com.b { requires com.a; exports com.b; opens com.b.resources to com.a; } package com.b; import com.a.ClassA; public class ClassB extends ClassA{ public ClassB(){ } } //------------ MODULE C module com.c { requires com.a; requires com.b; } package com.c; import com.b.ClassB; public class ModuleTest{ public static void main(String[] args){ new ClassB(); } } -Wolfgang > On 17/05/2017 18:22, wzberger wrote: > >> The resources are mainly images and XML files, located in separate >> packages. By adding the opens clause it works fine for Class B >> (called from module a) - however, it does not work for Class A >> (returns null). I wonder if this is the intended behavior? > I'm not aware of any bugs in this area. Can you expand a bit more on > "returns null" case. If the packages are open as you say then code in > both `a` and `b` should be able to locate the resources in `b`. > > -Alan > > From Alan.Bateman at oracle.com Thu May 18 08:29:40 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 09:29:40 +0100 Subject: How to load resources from base module In-Reply-To: References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> Message-ID: <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> On 18/05/2017 08:16, wzberger wrote: > : > > public class ClassA{ > public ClassA() { > //success > System.out.println(getClass().getResource("/com/b/resources/test.txt")); The test creates an instance of ClassB (extends ClassA) and so getClass() returns ClassB here, not ClassA. So when constructing a ClassB then this is equivalent to ClassB.class.getResource(...). > > //fail - null > System.out.println(ClassA.class.getResource("/com/b/resources/test.txt")); > Right, as this resource is not in the com.a module. Change this to ClassB.class.getResource(...) and it should work. -Alan From peter.levart at gmail.com Thu May 18 08:31:22 2017 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 18 May 2017 10:31:22 +0200 Subject: How to load resources from base module In-Reply-To: References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> Message-ID: Hi Wolfgang, You are using instance method Class#getResource(String) which states in javadoc: /If this class is in a named Module then this method will attempt to find the resource *in the module*./ "this class" is the class represented by the Class object which is the receiver of the instance method invocation. In your below case, when you invoke getClass().getResource(), "this class" represents the runtime class of "this" object (ClassB in your case, since you create an instance of ClassB), but when you invoke ClassA.class.getResource(), "this class" represents ClassA. So 1st attempt succeeds, bacause you are looking for a resource in the correct module (which contains it), while the 2nd attempt fails, because you are looking into the wrong module which doesn't contain the resource. Class#getResource(String) instance method, in case of "this class" being in a named module, delegates to special (new in JDK 9) module's class loader method: /This is done by delegating to the module's class loader findResource(String,String) method, invoking it with the module name and the absolute name of the resource./ In addition to looking for resource "only" in a particular module, standard resource encapsulation rules apply: // /Resources in named modules are subject to the rules for encapsulation specified in the Module getResourceAsStream method and so this method returns null when the resource is a non-".class" resource in a package that is not open to the caller's module./ So you are correctly opening the com.b.resources package to module com.a (which is the caller's module - the module containing the class with code invoking the Class::getResource(String) method). You just have to look up into the correct module too. Or if you don't want to bother with identifying the module where the resource lives but just with the "resource path" of the resource, then you can still use the ClassLoader#getResource(String) method, which will look up the resource in the class loader (with parents 1st delegation order) regardless of in which module the resource lives. Standard resource encapsulation rules still apply though. Regards, Peter On 05/18/2017 09:16 AM, wzberger wrote: > Here is a simple example which demonstrates the issue - still > something missing? > > //------------ MODULE A > module com.a { > exports com.a; > } > > package com.a; > > public class ClassA{ > public ClassA() { > //success > System.out.println(getClass().getResource("/com/b/resources/test.txt")); > > //fail - null > System.out.println(ClassA.class.getResource("/com/b/resources/test.txt")); > > } > } > > //------------ MODULE B > module com.b { > requires com.a; > exports com.b; > opens com.b.resources to com.a; > } > > package com.b; > > import com.a.ClassA; > > public class ClassB extends ClassA{ > public ClassB(){ > } > } > > //------------ MODULE C > module com.c { > requires com.a; > requires com.b; > } > > package com.c; > > import com.b.ClassB; > > public class ModuleTest{ > public static void main(String[] args){ > new ClassB(); > } > } > > -Wolfgang > >> On 17/05/2017 18:22, wzberger wrote: >> >>> The resources are mainly images and XML files, located in separate >>> packages. By adding the opens clause it works fine for Class B >>> (called from module a) - however, it does not work for Class A >>> (returns null). I wonder if this is the intended behavior? >> I'm not aware of any bugs in this area. Can you expand a bit more on >> "returns null" case. If the packages are open as you say then code in >> both `a` and `b` should be able to locate the resources in `b`. >> >> -Alan >> >> > From stephan.herrmann at berlin.de Thu May 18 08:59:09 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Thu, 18 May 2017 10:59:09 +0200 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: Remi, I see your proposal as a minimal compromise, avoiding the worst of difficulties, but I think we can do better. Trade-off: In all posts I could not find a real reason against escaping, aside from aesthetics. I don't see this as sufficient motivation for a less-then-perfect solution. Clarity: I'm still not completely following your explanations, partly because of the jargon you are using. I'll leave it to Alex to decide if he likes the idea that JLS would have to explain terms like dotted production. Compare this to just adding a few more rules to the grammar, where no hand-waving is needed for an explanation. No, I did not say that escaping is a pervasive change. I never said that the grammar for ordinary compilation units should be changed. If you like we only need to extend one rule for the scope of modular compilation units: Identifier. It can't get simpler. Completeness: I understand you as saying, module names cannot start with "transitive". Mind you, that every modifier that will be added to the grammar for modules in the future will cause conflicts for names that are now legal, and you won't have a means to resolve this. By contrast, we can use the escaping approach even to solve one more problem that has been briefly touched on this list before: Automatic modules suffer from the fact that some artifact names may have Java keywords in their name, which means that these artifacts simply cannot be used as automatic modules, right? Why not apply escaping also here? *Any* dot-separated sequence of words could be used as module name, as long as module references have a means to escape any keywords in that sequence. Suitability for implementation: As said, your proposal resolves one problem, but still IDE functionality suffers from restricted keywords, because scanning and parsing need more context information than normal. - Recovery after a syntax error will regress. - Scanning arbitrary regions of code is not possible. Remember: In an IDE code with syntax errors is the norm, not an exception, as the IDE provides functionality to work on incomplete code. Stephan On 18.05.2017 00:34, Remi Forax wrote: > I want to answer this before we start the meetings because i really think that restricted keyword as i propose solve the issues Stephan raised. > > > ----- Mail original ----- >> De: "Stephan Herrmann" >> ?: jigsaw-dev at openjdk.java.net >> Envoy?: Mardi 16 Mai 2017 11:49:45 >> Objet: Re: An alternative to "restricted keywords" > >> Thanks, Remi, for taking this to the EG list. >> >> Some collected responses: >> >> >> Remi: "from the user point of view, '^' looks like a hack" >> >> This is, of course, a subjective statement. I don't share this view >> and in years of experience with Xtext-languages (where this concept >> is used by default) I never heard any user complain about this. >> >> More importantly, I hold that such aesthetic considerations are of >> much lesser significance than the question, whether we can explain >> - unambiguously explain - the concept in a few simple sentences. >> Explaining must be possible at two levels: in a rigorous specification >> and in simple words for users of the language. > > I'm not against ^, or ` as it has already asked to escape an identifier, but as you said it's a pervasive change that applies on the whole grammar while i think that with restricted keyword (that really should be called local keywords) the changes only impact the grammar that specifies a module-info.java > >> >> Remi: "a keyword which is activated if you are at a position in the >> grammar where it can be recognized". >> >> I don't think 'being at a position in the grammar' is a good way of >> explaining. Parsing doesn't generally have one position in a grammar, >> multiple productions can be active in the same parser state. >> Also speaking of a "loop" for modifiers seems to complicate matters >> more than necessary. >> >> Under these considerations I still see '^' as the clearest of all >> solutions. Clear as a specification, simple to explain to users. > > Eclipse uses a LR parser, for a LR parser, position == dotted production as i have written earlier, so no problem because it corresponds to only one parser state. Note that even if one do not use an LR or a LL parser, most hand written parser i've seen, javac is one of them, also refers to dotted production in the comments of the corresponding methods. > >> >> >> >> Peter spoke about module names vs. package names. >> >> I think we agree, that module names cannot use "module words", >> whereas package names should be expected to contain them. > > yes, that the main issue, package names may contains unqualified name like 'transitive, ''with' or 'to'. > but i think people will also want to use existing package or more exactly prefix of existing package as module name, so we should also support having restricted keyword name as part of a module name. > > The grammar is: > > open? module module_name { > requires (transitive | static)* module_name; > exports package_name; > exports package_name to module_name1, module_name2; > opens package_name; > opens package_name to module_name1, module_name2; > uses xxx; > provides xxx with xxx, yyy; > } > > If we just consider package name, only 'opens' and 'exports' are followed by a package name and a package name can only been followed by ';' or 'to', so once 'opens' is parsed, you know that you can have only an identifier so if it's not an identifier by one of the restricted keywords, it should be considered as an identifier. > > As i said earlier, the scanner can see the restricted keyword as keyword and before feeding the token to the parser, you can check the parser state to see if the keyword as to be lowered to an identifier or not. > > For module name, there is the supplementary problem of transitive, because if a module starts with transitive, you can have a conflict. As i said earlier, instead of using the next token to know if transitive is the keyword or part of the module name, i think we should consider it as a keyword, as the JLS said a restricted keyword is activated when it can appear, so "requires transitive" is not a valid directive. > >> >> Remi: "you should use reverse DNS naming for package so no problem :)" >> >> "to" is a "module word" and a TLD. >> I think we should be very careful in judging that a existing conflict >> is not a real problem. Better to clearly and rigorously avoid the >> conflict in the first place. > > to as the first part of a package/module and to as in exports ... to can not be present on the same dotted production, because exports as to be followed by a package_name so 'to' here means the start of a package name and then because a package name can not ends with '.' you always know if you are inside the production recognizing the package_name or outside matching the to of the directive exports. > >> >> >> >> Some additional notes from my side: >> >> In the escape-approach, it may be prudent to technically allow >> escaping even words that are identifiers in Java 9, but could become >> keywords in a future version. This ensures that modules which need >> more escaping in Java 9+X can still be parsed in Java 9. > > yes, that's why i think that escaping is not the right mechanism here, because we want to solve a very local problem so we do not need a global grammar-wise way to solve our problem. > >> >> >> Current focus was on names of modules, packages and types. >> A complete solution must also give an answer for annotations on modules. >> Some possible solutions: >> a. Assume that annotations for modules are designed with modules in mind >> and thus have to avoid any module words in their names. >> b. Support escaping also in annotations >> c. Refine the scope where "module words" are keywords, let it start only >> when the word "module" or the group "open module" has been consumed. >> This would make the words "module" and "open" special, as being >> switch words, where we switch from one language to another. >> (For this I previously coined the term "scoped keywords" [1]) > > For annotation, again, because annotation name are qualified, you now when you see 'module' if you are in the middle of the annotation name or if you are outside. > >> >> >> I think we all agree that the conflicts we are solving here are rare >> corner cases. Most names do not contain module words. Still, from a >> conceptual and technical p.o.v. the solution must be bullet proof. >> But there's no need to be afraid of module declarations being spammed >> with dozens of '^' characters. Realistically, this will not happen. >> > > I agree, and i strongly believe that scoped keyword, local keywords or restricted keywords, i.e. whatever the name, keywords that are keywords or identifiers depending on the parser state are the general mechanism that solve our problem. > >> Stephan >> >> [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 >> > > R?mi > From dalibor.topic at oracle.com Thu May 18 11:45:26 2017 From: dalibor.topic at oracle.com (dalibor topic) Date: Thu, 18 May 2017 13:45:26 +0200 Subject: Some suggested patches and improvements In-Reply-To: <1E02E868-9599-43CC-A099-9853FBE2C5F7@cox.net> References: <81584479-dcc2-c96a-13e7-ef9f0010bcee@redhat.com> <1E02E868-9599-43CC-A099-9853FBE2C5F7@cox.net> Message-ID: On 18.05.2017 02:29, Gregg Wonderly wrote: > I understand that you might feel this is an excessive rant. [snip] > I will be looking at Swift and/or Go and just moving on. All the best on your future technological adventures, in any case, whether they are with Java, or something else! Please do keep in mind that this list is for "Technical discussion about Project Jigsaw", per http://mail.openjdk.java.net/mailman/listinfo/jigsaw-dev . If you find that the technical focus poses a challenge for you, please try to finding a more suitable outlet for your writings. cheers, dalibor topic -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From aguibert at us.ibm.com Thu May 18 13:40:52 2017 From: aguibert at us.ibm.com (Andrew Guibert) Date: Thu, 18 May 2017 08:40:52 -0500 Subject: Add-Reads and Add-Modules manifest attributes? Message-ID: I was happy to see that a resolution was reached for #AddExportsInManifest, but why did this stop at Add-Exports and Add-Opens? In order to cover all use cases (for unnamed modules), we would need two more manifest attributes: # Equivalent to: --add-reads=java.base=ALL-UNNAMED --add-reads=java.sql=ALL-UNNAMED Add-Reads: java.base, java.sql # Equivalent to --add-modules=java.se.ee Add-Modules: java.se.ee Based on my experience with JPMS so far, manifest attributes are a much cleaner way of specifying the JPMS workaround options. Mainly because on older JDKs they are silently ignored so I don't have to add conditionally apply the new Java 9 args in my launch commands. - Andy From Alan.Bateman at oracle.com Thu May 18 13:44:27 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 14:44:27 +0100 Subject: Add-Reads and Add-Modules manifest attributes? In-Reply-To: References: Message-ID: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> On 18/05/2017 14:40, Andrew Guibert wrote: > > I was happy to see that a resolution was reached for #AddExportsInManifest, > but why did this stop at Add-Exports and Add-Opens? In order to cover all > use cases (for unnamed modules), we would need two more manifest > attributes: > > # Equivalent to: --add-reads=java.base=ALL-UNNAMED > --add-reads=java.sql=ALL-UNNAMED > Add-Reads: java.base, java.sql > Unnamed modules read all modules so you don't need this. -Alan. From reto.merz at abacus.ch Thu May 18 13:46:34 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Thu, 18 May 2017 15:46:34 +0200 Subject: Add-Reads and Add-Modules manifest attributes? Message-ID: Yes that would be great and I have also asked for that, see this thread: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012138.html Regards Reto > -----Urspr?ngliche Nachricht----- > Von: jigsaw-dev [mailto:jigsaw-dev-bounces at openjdk.java.net] Im Auftrag > von Andrew Guibert > Gesendet: Donnerstag, 18. Mai 2017 15:41 > An: jigsaw-dev > Betreff: Add-Reads and Add-Modules manifest attributes? > > > > I was happy to see that a resolution was reached for #AddExportsInManifest, > but why did this stop at Add-Exports and Add-Opens? In order to cover all > use cases (for unnamed modules), we would need two more manifest > attributes: > > # Equivalent to: --add-reads=java.base=ALL-UNNAMED > --add-reads=java.sql=ALL-UNNAMED > Add-Reads: java.base, java.sql > > # Equivalent to --add-modules=java.se.ee > Add-Modules: java.se.ee > > Based on my experience with JPMS so far, manifest attributes are a much > cleaner way of specifying the JPMS workaround options. Mainly because on > older JDKs they are silently ignored so I don't have to add conditionally > apply the new Java 9 args in my launch commands. > > - Andy From aguibert at us.ibm.com Thu May 18 14:14:10 2017 From: aguibert at us.ibm.com (Andrew Guibert) Date: Thu, 18 May 2017 09:14:10 -0500 Subject: Add-Reads and Add-Modules manifest attributes? In-Reply-To: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> References: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> Message-ID: > From: Alan Bateman > To: Andrew Guibert , jigsaw-dev dev at openjdk.java.net> > Date: 05/18/2017 08:44 AM > Subject: Re: Add-Reads and Add-Modules manifest attributes? > > On 18/05/2017 14:40, Andrew Guibert wrote: > > > > > I was happy to see that a resolution was reached for #AddExportsInManifest, > > but why did this stop at Add-Exports and Add-Opens? In order to cover all > > use cases (for unnamed modules), we would need two more manifest > > attributes: > > > > # Equivalent to: --add-reads=java.base=ALL-UNNAMED > > --add-reads=java.sql=ALL-UNNAMED > > Add-Reads: java.base, java.sql > > > Unnamed modules read all modules so you don't need this. Regarding Add-Reads: You're reading it backwards I'm afraid =) This example is saying "java.base needs to read ALL-UNNAMED modules" But regardless of what example was given, it is a valid use case right? Regarding Add-Modules: I see that Reto made a similar request last week for --add-modules as a manifest attribute. You mentioned that there is no equivalent to `--add-modules` in executable JARs along with some backing evidence that it needs to be used in conjunction with `--module-path` and such. However, couldn't the JVM make a best effort to read and process an `Add-Modules` attr? I would argue that a vast majority of cases will be adding one of basic values (i.e. java.se.ee or ALL-SYSTEM) which do not need to be used in conjunction with `--module-path`. From russell.gold at oracle.com Thu May 18 14:20:34 2017 From: russell.gold at oracle.com (Russell Gold) Date: Thu, 18 May 2017 10:20:34 -0400 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> References: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> Message-ID: Maven support and tool support in general for MR Jars is very poor at the moment - including the bundle plugin. I do have a working example you could look at that includes OSGi support. The key here is that I delete the java9 classes before computing the OSGi manifest, and only compile them before building the jar. Russ > On May 17, 2017, at 11:28 PM, Ralph Goers wrote: > > I am afraid I have to echo these sentiments to some degree. In trying to get Log4j to support Java 9 I first tried to use a multi-release jar. This failed miserably when the OSGi build tool failed over finding java classes under META-INF. Then it proceeded to complain about the module-info.java files. Why these are java syntax instead of json or something more sensible for something that only contains declarations is a mystery to me. FWIW - the OSGi people don?t seem interested in supporting these new features - https://issues.apache.org/jira/browse/FELIX-5592 . > > I have been able to work around some of these issues but it has made the Log4j build very fragile and I haven?t really begun to see what happens when log4j is actually modularized or runs in an application that is. > > Ralph > >> On May 17, 2017, at 10:26 AM, Eric Johnson wrote: >> >> On Wed, May 17, 2017 at 1:08 AM, Andrew Dinn wrote: >> >>> On 16/05/17 19:11, Gregg Wonderly wrote: >>> >>> >>> >>>> If we really cannot actually keep from breaking 90% of existing Java >>>> in the market place when this new JDK release goes out, how valuable >>>> is JigSaw really? >>> >>> citation needed? >>> >> >> I mostly ignore jigsaw, and check in every now and then. >> >> I have a few co-workers that have poked at migrating their products to Java >> 9. So far as I know, nobody has succeeded yet. >> >> With significant regularity, I see issues pop up on this list that have odd >> problems, or persist in being unresolved. One of my favorites at the moment >> is automatic module names - a problem that Jigsaw caused for itself. Maybe >> that one is resolved for now, but I'm pretty certain that questions will >> come flooding back once Java 9 GAs. >> >> As near as I can tell, applications that compile and run under Java 8 will >> mostly *not* "just work" with Java 9 JRE. And that seems to be the lived >> experience of my co-workers. If a project is lucky, the only changes >> necessary will involve command line parameters. If a team is unlucky, they >> will need to rebuild for Java 9. If a team is really unlucky, they will >> need to partially or fully modularize. At which point some even more >> juggling is required to continue to support Java 7 & 8, if that's required >> by customers. >> >> My overall concerns for Jigsaw: >> https://medium.com/@one.eric.johnson/java-9-jigsaw-troubles-4fc406ef41e0 >> >> I'm not sure what citations you expect to see. There's probably nobody out >> there who can afford to pre-flight an EA build of Java 9 against all their >> products to see what the actual costs are going to be. Based on anecdotal >> evidence from this mailing list, significant players in the Java ecosystem >> - build tools, IDEs, critical libraries - have all had to fix unexpected >> breakages with Java 9. Obviously, the ones that don't break don't typically >> show up, so this is a self-selecting example, but an important one. >> >> However, even something as simple as requiring changes to command line >> parameters in order to launch a program compiled for Java 8 is a breaking >> change. The Jigsaw team seems to be taking this as a mere complaint, rather >> than as a genuine compatibility issue. >> >> Here's a challenge back to the Jigsaw team. Can I still do java -jar ... >> every existing Java application (without recompile!) that currently >> launches that way? I'm even willing to cut some slack and ignore >> applications that use com.sun APIs that have been "private" for years. Will >> that still work? The Jigsaw community should be able to provide evidence >> that's still possible, not that we should be required to provide evidence >> that it isn't. >> >> Eric. >> >> >>> regards, >>> >>> >>> Andrew Dinn >>> ----------- >>> >> > From rafael.wth at gmail.com Thu May 18 14:20:59 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Thu, 18 May 2017 16:20:59 +0200 Subject: Attaching to a JVM image that does not include java.instrument Message-ID: Hei, I found that it is impossible to dynamically attach to a JVM that does not include the java.instrument module when built with jlink. This is a result of the instrumentation API and its infrastructure missing from the image. rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar -m com.greetings/com.greetings.Main Error occurred during initialization of VM Could not find agent library instrument on the library path, with error: libinstrument.so: cannot open shared object file: No such file or directory It is of course neither possible to attach an agent to the JVM on the command line. This problem resolves when explicitly including the java.instrumentation module what does however render an unused warning in most IDEs. I understand that one target of the jlink tool is to reduce the JVM's size. For the example greetings app, the size different is however minimal: rafael at rafc:~/jdk9-test$ du -sh greetingsapp-instrument 45M greetingsapp-instrument rafael at rafc:~/jdk9-test$ du -sh greetingsapp 44M greetingsapp I would therefore suggest to always include the instrumentation module when building with jlink.I would find it rather confusing to end users not being able to attach an agent more or less all existing tooling demands this ability, especially since users of Java applications are not always responsible for building applications. At a minimum, I would suggest a better error message and the rejection of the javaagent parameter on the bundled command java executable. However, I strongly suggest to not go for this solution as the instrumentation module is unlikely to be included by any main application what would result in such images never being compatible with agents, this breaking compatibility of jlink built images to basically any Java tooling including that provided by Oracle. Best regards, Rafael From Alan.Bateman at oracle.com Thu May 18 14:27:15 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 15:27:15 +0100 Subject: Add-Reads and Add-Modules manifest attributes? In-Reply-To: References: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> Message-ID: <90d41cdb-5014-999e-e516-28ed100a9edb@oracle.com> On 18/05/2017 15:14, Andrew Guibert wrote: > : > > Regarding Add-Reads: > You're reading it backwards I'm afraid =) This example is saying > "java.base needs to read ALL-UNNAMED modules" > But regardless of what example was given, it is a valid use case right? > I assumed you had transposed them in your mail :-) In any case, it's not clear that you need this, at least not for the platform modules. Core reflection assumes readability so there shouldn't be any issues with code in either module reflecting into code on the class path. Maybe you are spinning bytecode? If so then you have a visibility issue as the types on the class path are not visible to boot or platform loaders. At a stretch then maybe are spinning bytecode that uses method handles, that's the only scenario that I can think of where you might need to add a read edge at runtime, in which case you can just call the reflective APIs. So maybe you could help by giving a specific example where you need this. -Alan From mark.reinhold at oracle.com Thu May 18 14:48:40 2017 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 18 May 2017 07:48:40 -0700 (PDT) Subject: Proposal: Allow illegal reflective access by default in JDK 9 Message-ID: <20170518144840.DE396D342B@eggemoggin.niobe.net> Over time, as we've gotten closer and closer to the JDK 9 GA date, more and more developers have begun paying attention to the actual changes in this release. The strong encapsulation of JDK-internal APIs has, in particular, triggered many worried expressions of concern that code that works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of this change was given in JDK 8. To help the entire ecosystem migrate to the modular Java platform at a more relaxed pace I hereby propose to allow illegal reflective access from code on the class path by default in JDK 9, and to disallow it in a future release. In short, the existing "big kill switch" of the `--permit-illegal-access` option [1] will become the default behavior of the JDK 9 run-time system, though without as many warnings. The current behavior of JDK 9, in which illegal reflective-access operations from code on the class path are not permitted, will become the default in a future release. Nothing will change at compile time. In detail, the recently-introduced `--permit-illegal-access` option will be replaced by a more-general option, `--illegal-access`. This option will take a single keyword parameter, as follows: `--illegal-access=permit` This will be the default mode for JDK 9. It opens every package in every explicit module to code in all unnamed modules, i.e., code on the class path, just as `--permit-illegal-access` does today. The first illegal reflective-access operation causes a warning to be issued, as with `--permit-illegal-access`, but no warnings are issued after that point. This single warning will describe how to enable further warnings. `--illegal-access=warn` This causes a warning message to be issued for each illegal reflective-access operation. This is equivalent to the current `--permit-illegal-access` option. `--illegal-access=debug` This causes both a warning message and a stack trace to be shown for each illegal reflective-access operation. This is equivalent to combining today's `--permit-illegal-access` option with `-Dsun.reflect.debugModuleAccessChecks`. `--illegal-access=deny` This disables all illegal reflective-access operations except for those enabled by other command-line options, such as `--add-opens`. This will become the default mode in a future release. Notes: - The proposed default mode enables the run-time system to issue a warning message, possibly at some time long after startup, without having been explicitly requested to do so. This may be a surprise in production environments, since it's extremely unusual for the run-time system to issue any warning messages at all. If the default mode permits illegal reflective access, however, then it's essential to make that known so that people aren't surprised when this is no longer the default mode in a future release. - Warning messages in any mode can be avoided, as before, by the judicious use of the `--add-exports` and `--add-opens` options. - This proposal will, if adopted, require adjustments to JEP 260, "Encapsulate Most Internal APIs" [2]. APIs that are internal to the JDK will still be strongly encapsulated from the standpoint of code in modules, whether those modules are automatic or explicit, but they will not appear to be encapsulated at run time from the standpoint of code on the class path. - When `deny` becomes the default mode then I expect `permit` to remain supported for at least one release, so that developers can continue to migrate their code. The `permit`, `warn`, and `debug` modes will, over time, be removed, as will the `--illegal-access` option itself. (For launch-script compatibility the unsupported modes will most likely just be ignored, after issuing a warning to that effect.) - This change will not magically solve every JDK 9 adoption problem. The concrete types of the built-in class loaders are still different, `rt.jar` is still gone, the layout of a system image is still not the same, and the version string still has a new format. Comments? - Mark [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html [2] http://openjdk.java.net/jeps/260 From aph at redhat.com Thu May 18 15:18:29 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 18 May 2017 16:18:29 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: On 18/05/17 15:48, mark.reinhold at oracle.com wrote: > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. Mmm. There are many scripts which parse the output of java,and many would break. Might '--illegal-access=permit,quiet' be worth considering? Andrew. From Alan.Bateman at oracle.com Thu May 18 15:19:11 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 16:19:11 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: On 18/05/2017 15:20, Rafael Winterhalter wrote: > Hei, > > I found that it is impossible to dynamically attach to a JVM that does not > include the java.instrument module when built with jlink. This is a result > of the instrumentation API and its infrastructure missing from the image. > > rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java > -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar > -m com.greetings/com.greetings.Main > Error occurred during initialization of VM > > Could not find agent library instrument on the library path, with error: > libinstrument.so: cannot open shared object file: No such file or directory Ugh, when -javaagent is specified then it is supposed to work as if `--add-modules java.instrument` is also on the command line. This ensures that the java.instrument module is resolved. The intention was also to get a useful error if the run-time image doesn't contain java.instrument. Another example is `-Dcom.sun.management.*` which should work as if `--add-modules jdk.management.agent` is specified on the command line. Also `-XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI` which is supposed to add `--add-modules jdk.internal.vm.ci`. I just checked both of these and the error is helpful as I expected. As regards the attach mechanism then it will work when it is compiled in and enabled in the target VM. When I say "compiled in" then I mean the support is in libjvm. When using jlink when you can select the minimal VM to get a much smaller VM that doesn't have the serviceability features. That won't be typical of course so assume it is libjvm then you should be able to attach and troubleshooting tools such as `jcmd` will work. If you attempt to load a java agent then the target VM will attempt to resolve and load the java.instrument module and so will fail if the run-time image doesn't have the module. You are right that java.instrument is small but it's always been a goal to be able to create run-time images that only contain java.base. I didn't understand your comment about an "unused warnings" in the IDE. Do you mean `requires java.instrument` rather than specifying java.instrument to the `jlink --add-modules` option? -Alan From cedric.champeau at gmail.com Thu May 18 15:22:11 2017 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Thu, 18 May 2017 17:22:11 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: This looks like a very reasonable proposal, +1 2017-05-18 16:48 GMT+02:00 : > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017- > March/011763.html > [2] http://openjdk.java.net/jeps/260 > From rgoers at apache.org Thu May 18 15:38:32 2017 From: rgoers at apache.org (Ralph Goers) Date: Thu, 18 May 2017 08:38:32 -0700 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: References: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> Message-ID: <94506876-1BAE-4228-B828-DE1B63ABBDD4@apache.org> Yes, I had to do something similar. But this is fragile if you don?t always do a clean or explicitly include a clean step that runs to clean up those items even if you didn?t specify it. Ralph > On May 18, 2017, at 7:20 AM, Russell Gold wrote: > > Maven support and tool support in general for MR Jars is very poor at the moment - including the bundle plugin. I do have a working example you could look at that includes OSGi support. The key here is that I delete the java9 classes before computing the OSGi manifest, and only compile them before building the jar. > > Russ > >> On May 17, 2017, at 11:28 PM, Ralph Goers > wrote: >> >> I am afraid I have to echo these sentiments to some degree. In trying to get Log4j to support Java 9 I first tried to use a multi-release jar. This failed miserably when the OSGi build tool failed over finding java classes under META-INF. Then it proceeded to complain about the module-info.java files. Why these are java syntax instead of json or something more sensible for something that only contains declarations is a mystery to me. FWIW - the OSGi people don?t seem interested in supporting these new features - https://issues.apache.org/jira/browse/FELIX-5592 >. >> >> I have been able to work around some of these issues but it has made the Log4j build very fragile and I haven?t really begun to see what happens when log4j is actually modularized or runs in an application that is. >> >> Ralph >> >>> On May 17, 2017, at 10:26 AM, Eric Johnson > wrote: >>> >>> On Wed, May 17, 2017 at 1:08 AM, Andrew Dinn > wrote: >>> >>>> On 16/05/17 19:11, Gregg Wonderly wrote: >>>> >>>> >>>> >>>>> If we really cannot actually keep from breaking 90% of existing Java >>>>> in the market place when this new JDK release goes out, how valuable >>>>> is JigSaw really? >>>> >>>> citation needed? >>>> >>> >>> I mostly ignore jigsaw, and check in every now and then. >>> >>> I have a few co-workers that have poked at migrating their products to Java >>> 9. So far as I know, nobody has succeeded yet. >>> >>> With significant regularity, I see issues pop up on this list that have odd >>> problems, or persist in being unresolved. One of my favorites at the moment >>> is automatic module names - a problem that Jigsaw caused for itself. Maybe >>> that one is resolved for now, but I'm pretty certain that questions will >>> come flooding back once Java 9 GAs. >>> >>> As near as I can tell, applications that compile and run under Java 8 will >>> mostly *not* "just work" with Java 9 JRE. And that seems to be the lived >>> experience of my co-workers. If a project is lucky, the only changes >>> necessary will involve command line parameters. If a team is unlucky, they >>> will need to rebuild for Java 9. If a team is really unlucky, they will >>> need to partially or fully modularize. At which point some even more >>> juggling is required to continue to support Java 7 & 8, if that's required >>> by customers. >>> >>> My overall concerns for Jigsaw: >>> https://medium.com/@one.eric.johnson/java-9-jigsaw-troubles-4fc406ef41e0 >>> >>> I'm not sure what citations you expect to see. There's probably nobody out >>> there who can afford to pre-flight an EA build of Java 9 against all their >>> products to see what the actual costs are going to be. Based on anecdotal >>> evidence from this mailing list, significant players in the Java ecosystem >>> - build tools, IDEs, critical libraries - have all had to fix unexpected >>> breakages with Java 9. Obviously, the ones that don't break don't typically >>> show up, so this is a self-selecting example, but an important one. >>> >>> However, even something as simple as requiring changes to command line >>> parameters in order to launch a program compiled for Java 8 is a breaking >>> change. The Jigsaw team seems to be taking this as a mere complaint, rather >>> than as a genuine compatibility issue. >>> >>> Here's a challenge back to the Jigsaw team. Can I still do java -jar ... >>> every existing Java application (without recompile!) that currently >>> launches that way? I'm even willing to cut some slack and ignore >>> applications that use com.sun APIs that have been "private" for years. Will >>> that still work? The Jigsaw community should be able to provide evidence >>> that's still possible, not that we should be required to provide evidence >>> that it isn't. >>> >>> Eric. >>> >>> >>>> regards, >>>> >>>> >>>> Andrew Dinn >>>> ----------- >>>> >>> >> > From aguibert at us.ibm.com Thu May 18 15:47:20 2017 From: aguibert at us.ibm.com (Andrew Guibert) Date: Thu, 18 May 2017 10:47:20 -0500 Subject: Add-Reads and Add-Modules manifest attributes? In-Reply-To: <90d41cdb-5014-999e-e516-28ed100a9edb@oracle.com> References: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> <90d41cdb-5014-999e-e516-28ed100a9edb@oracle.com> Message-ID: > From: Alan Bateman > To: Andrew Guibert > Cc: jigsaw-dev > Date: 05/18/2017 09:27 AM > Subject: Re: Add-Reads and Add-Modules manifest attributes? > > On 18/05/2017 15:14, Andrew Guibert wrote: > > Regarding Add-Reads: > You're reading it backwards I'm afraid =) This example is saying > "java.base needs to read ALL-UNNAMED modules" > But regardless of what example was given, it is a valid use case right? > I assumed you had transposed them in your mail :-) > > In any case, it's not clear that you need this, at least not for the > platform modules. Core reflection assumes readability so there > shouldn't be any issues with code in either module reflecting into > code on the class path. Maybe you are spinning bytecode? If so then > you have a visibility issue as the types on the class path are not > visible to boot or platform loaders. At a stretch then maybe are > spinning bytecode that uses method handles, that's the only scenario > that I can think of where you might need to add a read edge at > runtime, in which case you can just call the reflective APIs. So > maybe you could help by giving a specific example where you need this. A concrete use case for an `Add-Reads` manifest attr is beside the point. Since we have the command line option, why can't we have the equivalent functionality as a manifest attribute (especially since it seems trivial to implement)? As long as use cases exist for `--add-reads`, use cases will exist for the manifest attribute. If no use cases exist for the `--add-reads` command line option, can it be removed entirely? Also, I don't want to forget about the discussion for `Add-Modules`. I understand that it may pose a significant technical challenge, but for code launched via `java -jar` which depends on `java.se.ee` specific modules it is a must for compatibility. From russell.gold at oracle.com Thu May 18 16:02:45 2017 From: russell.gold at oracle.com (Russell Gold) Date: Thu, 18 May 2017 12:02:45 -0400 Subject: Evidence of incompatibility (was Some suggested patches and improvements) In-Reply-To: <94506876-1BAE-4228-B828-DE1B63ABBDD4@apache.org> References: <18008B28-1846-4192-9595-99AC2D1DBC6E@apache.org> <94506876-1BAE-4228-B828-DE1B63ABBDD4@apache.org> Message-ID: No, the cleanup of the java 9 classes is bound to the compile step. Therefore, you don?t need to do a clean first. That would have been very aggravating. > On May 18, 2017, at 11:38 AM, Ralph Goers wrote: > > Yes, I had to do something similar. But this is fragile if you don?t always do a clean or explicitly include a clean step that runs to clean up those items even if you didn?t specify it. > > Ralph > >> On May 18, 2017, at 7:20 AM, Russell Gold > wrote: >> >> Maven support and tool support in general for MR Jars is very poor at the moment - including the bundle plugin. I do have a working example you could look at that includes OSGi support. The key here is that I delete the java9 classes before computing the OSGi manifest, and only compile them before building the jar. >> >> Russ >> >>> On May 17, 2017, at 11:28 PM, Ralph Goers > wrote: >>> >>> I am afraid I have to echo these sentiments to some degree. In trying to get Log4j to support Java 9 I first tried to use a multi-release jar. This failed miserably when the OSGi build tool failed over finding java classes under META-INF. Then it proceeded to complain about the module-info.java files. Why these are java syntax instead of json or something more sensible for something that only contains declarations is a mystery to me. FWIW - the OSGi people don?t seem interested in supporting these new features - https://issues.apache.org/jira/browse/FELIX-5592 >. >>> >>> I have been able to work around some of these issues but it has made the Log4j build very fragile and I haven?t really begun to see what happens when log4j is actually modularized or runs in an application that is. >>> >>> Ralph >>> >>>> On May 17, 2017, at 10:26 AM, Eric Johnson > wrote: >>>> >>>> On Wed, May 17, 2017 at 1:08 AM, Andrew Dinn > wrote: >>>> >>>>> On 16/05/17 19:11, Gregg Wonderly wrote: >>>>> >>>>> >>>>> >>>>>> If we really cannot actually keep from breaking 90% of existing Java >>>>>> in the market place when this new JDK release goes out, how valuable >>>>>> is JigSaw really? >>>>> >>>>> citation needed? >>>>> >>>> >>>> I mostly ignore jigsaw, and check in every now and then. >>>> >>>> I have a few co-workers that have poked at migrating their products to Java >>>> 9. So far as I know, nobody has succeeded yet. >>>> >>>> With significant regularity, I see issues pop up on this list that have odd >>>> problems, or persist in being unresolved. One of my favorites at the moment >>>> is automatic module names - a problem that Jigsaw caused for itself. Maybe >>>> that one is resolved for now, but I'm pretty certain that questions will >>>> come flooding back once Java 9 GAs. >>>> >>>> As near as I can tell, applications that compile and run under Java 8 will >>>> mostly *not* "just work" with Java 9 JRE. And that seems to be the lived >>>> experience of my co-workers. If a project is lucky, the only changes >>>> necessary will involve command line parameters. If a team is unlucky, they >>>> will need to rebuild for Java 9. If a team is really unlucky, they will >>>> need to partially or fully modularize. At which point some even more >>>> juggling is required to continue to support Java 7 & 8, if that's required >>>> by customers. >>>> >>>> My overall concerns for Jigsaw: >>>> https://medium.com/@one.eric.johnson/java-9-jigsaw-troubles-4fc406ef41e0 >>>> >>>> I'm not sure what citations you expect to see. There's probably nobody out >>>> there who can afford to pre-flight an EA build of Java 9 against all their >>>> products to see what the actual costs are going to be. Based on anecdotal >>>> evidence from this mailing list, significant players in the Java ecosystem >>>> - build tools, IDEs, critical libraries - have all had to fix unexpected >>>> breakages with Java 9. Obviously, the ones that don't break don't typically >>>> show up, so this is a self-selecting example, but an important one. >>>> >>>> However, even something as simple as requiring changes to command line >>>> parameters in order to launch a program compiled for Java 8 is a breaking >>>> change. The Jigsaw team seems to be taking this as a mere complaint, rather >>>> than as a genuine compatibility issue. >>>> >>>> Here's a challenge back to the Jigsaw team. Can I still do java -jar ... >>>> every existing Java application (without recompile!) that currently >>>> launches that way? I'm even willing to cut some slack and ignore >>>> applications that use com.sun APIs that have been "private" for years. Will >>>> that still work? The Jigsaw community should be able to provide evidence >>>> that's still possible, not that we should be required to provide evidence >>>> that it isn't. >>>> >>>> Eric. >>>> >>>> >>>>> regards, >>>>> >>>>> >>>>> Andrew Dinn >>>>> ----------- >>>>> >>>> >>> >> > From uschindler at apache.org Thu May 18 16:08:21 2017 From: uschindler at apache.org (Uwe Schindler) Date: Thu, 18 May 2017 16:08:21 +0000 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <5187E365-A606-4647-94F7-797C9C15FA7A@apache.org> Hi Mark, To me this proposal is a Desaster. I'd not do this. Buggy software may use the big kill switch. Sorry Red Hat guys: that's what you triggered with your "no". Bravo! I am impressed! Sorry Gradle, the worst design in software about environment variables made the whole world again as unsafe as before. We will again see ongoing security updates in Java just fix fix holes that are opened by default. When I have read the mails yesterday, I thought: do you really want to build your software with such a broken tool and it's ecosystem? Can you not just tell the plug-in authors to fix their shit and fix your API to work correct? Amazon S3 software dilettantes: Fix your EC2 security software to not undermine the Java security system! I can bring many more: Don't do that in security relevant tools or build systems many people rely on! Today is the worst day in Java history. Uwe Am 18. Mai 2017 16:48:40 MESZ schrieb mark.reinhold at oracle.com: >Over time, as we've gotten closer and closer to the JDK 9 GA date, more >and more developers have begun paying attention the actual changes in >this release. The strong encapsulation of JDK-internal APIs has, in >particular, triggered many worried expressions of concern that code >that >works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance >warning of this change was given in JDK 8. > >To help the entire ecosystem migrate to the modular Java platform at a >more relaxed pace I hereby propose to allow illegal reflective access >from code on the class path by default in JDK 9, and to disallow it in >a future release. > >In short, the existing "big kill switch" of the >`--permit-illegal-access` >option [1] will become the default behavior of the JDK 9 run-time >system, >though without as many warnings. The current behavior of JDK 9, in >which >illegal reflective-access operations from code on the class path are >not >permitted, will become the default in a future release. Nothing will >change at compile time. > >In detail, the recently-introduced `--permit-illegal-access` option >will >be replaced by a more-general option, `--illegal-access`. This option >will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > >Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > >- When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > >Comments? > >- Mark > > >[1] >http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html >[2] http://openjdk.java.net/jeps/260 From Alan.Bateman at oracle.com Thu May 18 16:21:11 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 17:21:11 +0100 Subject: Add-Reads and Add-Modules manifest attributes? In-Reply-To: References: <8c58c325-c744-e7f3-9313-56cba827d4c6@oracle.com> <90d41cdb-5014-999e-e516-28ed100a9edb@oracle.com> Message-ID: <9a3e48b0-7b05-1642-9bb7-7cdb78d94904@oracle.com> On 18/05/2017 16:47, Andrew Guibert wrote: > : > > A concrete use case for an `Add-Reads` manifest attr is beside the > point. Since we have the command line option, why can't we have the > equivalent functionality as a manifest attribute (especially since it > seems trivial to implement)? As long as use cases exist for > `--add-reads`, use cases will exist for the manifest attribute. If no > use cases exist for the `--add-reads` command line option, can it be > removed entirely? > `--add-reads` is important on the command line, esp. for testing when a module is patched at compile-time and run-time to includes tests in the same module as the module under test. An equivalent Add-Reads in the manifest that updates an explicit module to read all unnamed modules is less clear and it would be useful to have an example or two. You are right that it would be trivial to add but that isn't a good reason to add it. > > Also, I don't want to forget about the discussion for `Add-Modules`. > I understand that it may pose a significant technical challenge, but > for code launched via `java -jar` which depends on `java.se.ee` > specific modules it is a must for compatibility. > It has come up a few times. -Alan From neugens.limasoftware at gmail.com Thu May 18 16:32:39 2017 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Thu, 18 May 2017 18:32:39 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <5187E365-A606-4647-94F7-797C9C15FA7A@apache.org> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <5187E365-A606-4647-94F7-797C9C15FA7A@apache.org> Message-ID: 2017-05-18 18:08 GMT+02:00 Uwe Schindler : > Hi Mark, > > To me this proposal is a Desaster. I'd not do this. Buggy software may use the big kill switch. > > Sorry Red Hat guys: that's what you triggered with your "no". Bravo! I am impressed! > > Sorry Gradle, the worst design in software about environment variables made the whole world again as unsafe as before. We will again see ongoing security updates in Java just fix fix holes that are opened by default. When I have read the mails yesterday, I thought: do you really want to build your software with such a broken tool and it's ecosystem? Can you not just tell the plug-in authors to fix their shit and fix your API to work correct? > > Amazon S3 software dilettantes: Fix your EC2 security software to not undermine the Java security system! I can bring many more: Don't do that in security relevant tools or build systems many people rely on! > > Today is the worst day in Java history. Hello Uwe, I appreciate your enthusiasm, but please let's try to keep this discussion focused. Cheers, Mario -- pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens Proud GNU Classpath developer: http://www.classpath.org/ OpenJDK: http://openjdk.java.net/projects/caciocavallo/ Please, support open standards: http://endsoftpatents.org/ From misterm at gmail.com Thu May 18 16:40:51 2017 From: misterm at gmail.com (Michael Nascimento) Date: Thu, 18 May 2017 13:40:51 -0300 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: Sounds like a well thought compromise, Mark. Real world applications need this and it's a very welcome move. Regards, Michael On Thu, May 18, 2017 at 11:48 AM, wrote: > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017- > March/011763.html > [2] http://openjdk.java.net/jeps/260 > From paul.deitel at deitel.com Thu May 18 17:35:39 2017 From: paul.deitel at deitel.com (Paul Deitel) Date: Thu, 18 May 2017 13:35:39 -0400 Subject: -Xdiag:resolver Message-ID: In earlier JDK 9-ea releases, there was the -Xdiag:resolver option to the java command so you could view the module resolution process at app startup. Has this been removed and replaced with --show-module-resolition option? The outputs don?t seem to be the same with this new option, so I am just confirming that I did not miss something. From Alan.Bateman at oracle.com Thu May 18 17:47:35 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 May 2017 18:47:35 +0100 Subject: -Xdiag:resolver In-Reply-To: References: Message-ID: <443d8803-77e8-5618-1381-512a603d738c@oracle.com> On 18/05/2017 18:35, Paul Deitel wrote: > In earlier JDK 9-ea releases, there was the -Xdiag:resolver option to the java command so you could view the module resolution process at app startup. > > Has this been removed and replaced with --show-module-resolition option? The outputs don?t seem to be the same with this new option, so I am just confirming that I did not miss something. > Yes, this was cleaned up recently and -Xdiag:resolver was temporarily an alias for `--show-module-resolution`. The reason for the clean-up is to make it easy to see why service provider modules are resolved, also to make the output easy to grep. -Alan From markus_keller at ch.ibm.com Thu May 18 19:01:26 2017 From: markus_keller at ch.ibm.com (Markus Keller) Date: Thu, 18 May 2017 21:01:26 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: The command-line argument and the default mode sound good. `--illegal-access=quiet` could be added for use by clients that cannot cope with additional warning messages. The second ingredient for broad compatibility with Java 8 would be an automatic --add-modules=ALL-SYSTEM. This could be implemented as an automatic compatibility mode in Java 9 that would only be active as long as there's no --modulepath on the command line. An alternative to automatic compatibility would be another sub-option for --illegal-access, so that affected end users only have one command-line argument for all accessibility concerns, e.g.: `--illegal-access=permit-all` Equivalent to `--illegal-access=permit --add-modules=ALL-SYSTEM` FYI: In the Eclipse launcher for the Oxygen (4.7) release, we've added forward-compatibility handling for --permit-illegal-access. Our deadline to revert/adjust that is May 30 ( https://bugs.eclipse.org/516911 ). Regards, Markus "jigsaw-dev" wrote on 2017-05-18 16:48:40: > From: mark.reinhold at oracle.com > To: jigsaw-dev at openjdk.java.net > Date: 2017-05-18 16:49 > Subject: Proposal: Allow illegal reflective access by default in JDK 9 > Sent by: "jigsaw-dev" > > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > [2] http://openjdk.java.net/jeps/260 > From mandy.chung at oracle.com Thu May 18 21:20:40 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:20:40 +0000 Subject: hg: jigsaw/jake: 16 new changesets Message-ID: <201705182120.v4ILKfla017252@aojmv0008.oracle.com> Changeset: 7583288d0802 Author: ihse Date: 2017-05-10 09:02 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/7583288d0802 8178278: Move Standard Algorithm Names document to specs directory Reviewed-by: erikj ! make/Javadoc.gmk Changeset: 4635fa3adac6 Author: erikj Date: 2017-05-10 18:27 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/4635fa3adac6 8179867: JDK9 b167: demos exist in JDK bundles Reviewed-by: ihse ! make/Bundles.gmk ! make/Main.gmk Changeset: e50b2825a070 Author: tbell Date: 2017-05-10 14:55 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/e50b2825a070 8180129: Bundles.gmk:181: *** unterminated call to function 'filter-out': missing ')'. Stop. Reviewed-by: erikj ! make/Bundles.gmk Changeset: d045961bbd5c Author: ihse Date: 2017-05-11 08:56 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/d045961bbd5c 8179105: Respect "include_in_docs" property from imported modules. Reviewed-by: mchung, erikj ! make/common/Modules.gmk Changeset: 4dfb05777cb6 Author: erikj Date: 2017-05-11 18:41 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/4dfb05777cb6 8180083: Adjust Jib and JDL configurations for 9 to support new generation Mach 5 Reviewed-by: tbell, ihse ! common/conf/jib-profiles.js Changeset: 503670294c7d Author: lana Date: 2017-05-11 18:10 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/503670294c7d Merge Changeset: f1ca22ebd70f Author: iignatyev Date: 2017-05-11 13:58 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/f1ca22ebd70f 8180037: move jdk.test.lib.InMemoryJavaCompiler to a separate package Reviewed-by: mseledtsov, vlivanov ! test/lib/RedefineClassHelper.java - test/lib/jdk/test/lib/InMemoryJavaCompiler.java + test/lib/jdk/test/lib/compiler/InMemoryJavaCompiler.java Changeset: 52fd3ed2536f Author: iignatyev Date: 2017-05-11 13:58 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/52fd3ed2536f 8180004: jdk.test.lib.DynamicVMOption should be moved to jdk.test.lib.management Reviewed-by: mseledtsov, vlivanov - test/lib/jdk/test/lib/DynamicVMOption.java + test/lib/jdk/test/lib/management/DynamicVMOption.java Changeset: 69879afbd98f Author: ihse Date: 2017-05-12 19:09 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/69879afbd98f 8175825: Stop including pubs repo Reviewed-by: erikj ! common/autoconf/generated-configure.sh ! common/autoconf/spec.gmk.in ! common/bin/hgforest.sh ! common/doc/building.html ! common/doc/testing.html + make/Docs.gmk - make/Javadoc.gmk ! make/Main.gmk Changeset: f6e365a5e489 Author: ihse Date: 2017-05-12 19:11 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/f6e365a5e489 8180281: --with-jtreg is broken for many use cases Reviewed-by: erikj ! common/autoconf/generated-configure.sh ! common/autoconf/toolchain.m4 ! make/RunTests.gmk Changeset: c569c88b7650 Author: mchung Date: 2017-05-12 13:29 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/c569c88b7650 8180208: Provide a new docs bundle page Reviewed-by: ihse, jjg ! make/Docs.gmk ! make/Main.gmk Changeset: 2878d95ace72 Author: ihse Date: 2017-05-15 16:34 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/2878d95ace72 8180342: Fixup path for jtreg Reviewed-by: erikj ! common/autoconf/generated-configure.sh ! common/autoconf/toolchain.m4 Changeset: 9f46e4fbae64 Author: ihse Date: 2017-05-16 14:13 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/9f46e4fbae64 8180420: Set PATH for dot and pandoc in JIB Reviewed-by: erikj ! common/conf/jib-profiles.js Changeset: 4d163ec59d98 Author: ihse Date: 2017-05-16 14:14 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/4d163ec59d98 8180328: Bad links in footer of all javadoc-generated pages Reviewed-by: erikj ! make/Docs.gmk Changeset: 30196d168c55 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/30196d168c55 Added tag jdk-9+170 for changeset 4d163ec59d98 ! .hgtags Changeset: 38d57d90acbe Author: mchung Date: 2017-05-18 14:17 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/38d57d90acbe Merge ! .hgtags ! common/autoconf/generated-configure.sh ! common/autoconf/spec.gmk.in ! common/autoconf/toolchain.m4 ! common/bin/hgforest.sh ! common/conf/jib-profiles.js ! make/Docs.gmk ! make/Main.gmk ! make/common/Modules.gmk From mandy.chung at oracle.com Thu May 18 21:20:45 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:20:45 +0000 Subject: hg: jigsaw/jake/corba: 4 new changesets Message-ID: <201705182120.v4ILKjbS017359@aojmv0008.oracle.com> Changeset: a3e210a93f90 Author: jjg Date: 2017-05-10 15:24 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/a3e210a93f90 8180041: Fix HTML 5 issues in java.corba Reviewed-by: alanb, lancea ! src/java.corba/share/classes/com/sun/corba/se/spi/monitoring/package.html ! src/java.corba/share/classes/org/omg/CORBA/Any.java ! src/java.corba/share/classes/org/omg/CORBA/ORB.java ! src/java.corba/share/classes/org/omg/CORBA/doc-files/compliance.html ! src/java.corba/share/classes/org/omg/CORBA/doc-files/generatedfiles.html ! src/java.corba/share/classes/org/omg/CORBA/package.html ! src/java.corba/share/classes/org/omg/CORBA/portable/package.html ! src/java.corba/share/classes/org/omg/CORBA_2_3/package.html ! src/java.corba/share/classes/org/omg/PortableInterceptor/IOP.idl ! src/java.corba/share/classes/org/omg/PortableInterceptor/Interceptors.idl ! src/java.corba/share/classes/org/omg/PortableInterceptor/package.html ! src/java.corba/share/classes/org/omg/PortableServer/package.html Changeset: 8a4ab3b0ab9a Author: lana Date: 2017-05-11 18:10 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/8a4ab3b0ab9a Merge Changeset: c62e5964cfcf Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/c62e5964cfcf Added tag jdk-9+170 for changeset 8a4ab3b0ab9a ! .hgtags Changeset: bb3fec2e546e Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/bb3fec2e546e Merge ! .hgtags ! src/java.corba/share/classes/org/omg/CORBA/ORB.java From mandy.chung at oracle.com Thu May 18 21:20:52 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:20:52 +0000 Subject: hg: jigsaw/jake/hotspot: 8 new changesets Message-ID: <201705182120.v4ILKqmo017461@aojmv0008.oracle.com> Changeset: 830c73c1bd96 Author: ihse Date: 2017-05-11 09:00 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/830c73c1bd96 8174848: Remove gpl templates from hotspot/make Reviewed-by: erikj - make/templates/gpl-cp-header - make/templates/gpl-header Changeset: cab132bfdaec Author: lana Date: 2017-05-11 18:11 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/cab132bfdaec Merge - make/templates/gpl-cp-header - make/templates/gpl-header Changeset: bffc9b76c590 Author: iignatyev Date: 2017-05-11 14:03 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/bffc9b76c590 8180037: move jdk.test.lib.InMemoryJavaCompiler to a separate package Reviewed-by: mseledtsov, vlivanov ! test/gc/metaspace/TestMetaspacePerfCounters.java ! test/runtime/BadObjectClass/BootstrapRedefine.java ! test/runtime/RedefineTests/ModifyAnonymous.java ! test/runtime/Unsafe/DefineClass.java ! test/runtime/Unsafe/NestedUnsafe.java ! test/runtime/defineAnonClass/NestedUnsafe.java ! test/runtime/defineAnonClass/NestedUnsafe2.java ! test/runtime/getSysPackage/GetSysPkgTest.java ! test/runtime/modules/ModuleStress/ModuleStress.java ! test/runtime/modules/PatchModule/PatchModule2Dirs.java ! test/runtime/modules/PatchModule/PatchModuleCDS.java ! test/runtime/modules/PatchModule/PatchModuleClassList.java ! test/runtime/modules/PatchModule/PatchModuleJavaBase.java ! test/runtime/modules/PatchModule/PatchModuleTest.java ! test/runtime/modules/PatchModule/PatchModuleTestJar.java ! test/runtime/modules/PatchModule/PatchModuleTestJarDir.java ! test/runtime/modules/PatchModule/PatchModuleTraceCL.java ! test/runtime/modules/Visibility/PatchModuleVisibility.java ! test/runtime/modules/Visibility/XbootcpNoVisibility.java ! test/runtime/modules/Visibility/XbootcpVisibility.java Changeset: 434139d1c85b Author: iignatyev Date: 2017-05-11 14:13 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/434139d1c85b 8180004: jdk.test.lib.DynamicVMOption should be moved to jdk.test.lib.management Reviewed-by: mseledtsov, vlivanov ! test/gc/arguments/TestDynMaxHeapFreeRatio.java ! test/gc/arguments/TestDynMinHeapFreeRatio.java ! test/gc/parallel/TestDynShrinkHeap.java ! test/runtime/CommandLine/OptionsValidation/TestJcmdOutput.java ! test/runtime/CommandLine/OptionsValidation/common/optionsvalidation/JVMOption.java ! test/runtime/CommandLine/VMOptionsFile/TestVMOptionsFile.java Changeset: d6d7e5caf497 Author: tschatzl Date: 2017-05-15 12:20 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/d6d7e5caf497 8180048: Interned string and symbol table leak memory during parallel unlinking Summary: Make appending found dead BasicHashtableEntrys to the free list atomic. Reviewed-by: ehelin, shade, coleenp ! src/share/vm/classfile/stringTable.cpp ! src/share/vm/classfile/stringTable.hpp ! src/share/vm/classfile/symbolTable.cpp ! src/share/vm/classfile/symbolTable.hpp ! src/share/vm/runtime/vmStructs.cpp ! src/share/vm/utilities/hashtable.cpp ! src/share/vm/utilities/hashtable.hpp Changeset: 38a240fd58a2 Author: aph Date: 2017-05-11 13:11 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/38a240fd58a2 8179954: AArch64: C1 and C2 volatile accesses are not sequentially consistent Reviewed-by: roland ! src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp ! src/cpu/aarch64/vm/templateTable_aarch64.cpp Changeset: b3ee8ab233ed Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/b3ee8ab233ed Added tag jdk-9+170 for changeset 38a240fd58a2 ! .hgtags Changeset: 89f282fd426d Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/89f282fd426d Merge ! .hgtags ! src/share/vm/runtime/vmStructs.cpp ! src/share/vm/utilities/hashtable.hpp ! test/gc/metaspace/TestMetaspacePerfCounters.java ! test/gc/parallel/TestDynShrinkHeap.java ! test/runtime/BadObjectClass/BootstrapRedefine.java ! test/runtime/Unsafe/DefineClass.java ! test/runtime/getSysPackage/GetSysPkgTest.java ! test/runtime/modules/PatchModule/PatchModule2Dirs.java ! test/runtime/modules/PatchModule/PatchModuleCDS.java ! test/runtime/modules/PatchModule/PatchModuleJavaBase.java ! test/runtime/modules/PatchModule/PatchModuleTest.java ! test/runtime/modules/PatchModule/PatchModuleTestJar.java ! test/runtime/modules/PatchModule/PatchModuleTestJarDir.java ! test/runtime/modules/PatchModule/PatchModuleTraceCL.java ! test/runtime/modules/Visibility/PatchModuleVisibility.java ! test/runtime/modules/Visibility/XbootcpNoVisibility.java ! test/runtime/modules/Visibility/XbootcpVisibility.java From mandy.chung at oracle.com Thu May 18 21:20:55 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:20:55 +0000 Subject: hg: jigsaw/jake/jaxp: 5 new changesets Message-ID: <201705182120.v4ILKtvx017513@aojmv0008.oracle.com> Changeset: e79b44a9d2bd Author: joehw Date: 2017-05-09 18:26 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/e79b44a9d2bd 8179868: Java API Docs of javax.xml.transform.stax contains TODOs Reviewed-by: lancea, bpb ! src/java.xml/share/classes/javax/xml/transform/stax/package.html Changeset: 8861f89ae4c0 Author: lana Date: 2017-05-11 18:11 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/8861f89ae4c0 Merge Changeset: 6e78f902f477 Author: joehw Date: 2017-05-15 20:27 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/6e78f902f477 8180060: Examine copyright header for some files Reviewed-by: lancea ! src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/FeatureState.java ! src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/ParserConfigurationSettings.java ! src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/PropertyState.java ! src/java.xml/share/classes/com/sun/org/apache/xerces/internal/util/Status.java ! src/java.xml/share/classes/com/sun/xml/internal/stream/dtd/nonvalidating/XMLSimpleType.java Changeset: a5a3ac31f306 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/a5a3ac31f306 Added tag jdk-9+170 for changeset 6e78f902f477 ! .hgtags Changeset: 95172f5d5391 Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/95172f5d5391 Merge ! .hgtags From mandy.chung at oracle.com Thu May 18 21:20:57 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:20:57 +0000 Subject: hg: jigsaw/jake/jaxws: 2 new changesets Message-ID: <201705182120.v4ILKvoY017565@aojmv0008.oracle.com> Changeset: 139e7c786ee4 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/139e7c786ee4 Added tag jdk-9+170 for changeset e75d3abe579a ! .hgtags Changeset: eb19676ed32f Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/eb19676ed32f Merge ! .hgtags From mandy.chung at oracle.com Thu May 18 21:21:10 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:21:10 +0000 Subject: hg: jigsaw/jake/jdk: 28 new changesets Message-ID: <201705182121.v4ILLBM8017637@aojmv0008.oracle.com> Changeset: 3e92641a337d Author: psandoz Date: 2017-05-09 15:04 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/3e92641a337d 8177153: LambdaMetafactory has default constructor Reviewed-by: psandoz Contributed-by: ron.pressler at oracle.com ! src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java Changeset: 2c7616c8b7d8 Author: ihse Date: 2017-05-10 09:02 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/2c7616c8b7d8 8178278: Move Standard Algorithm Names document to specs directory Reviewed-by: erikj, wetmore, mullan ! src/java.base/share/classes/java/security/AlgorithmParameterGenerator.java ! src/java.base/share/classes/java/security/AlgorithmParameters.java ! src/java.base/share/classes/java/security/DrbgParameters.java ! src/java.base/share/classes/java/security/KeyFactory.java ! src/java.base/share/classes/java/security/KeyPairGenerator.java ! src/java.base/share/classes/java/security/KeyStore.java ! src/java.base/share/classes/java/security/MessageDigest.java ! src/java.base/share/classes/java/security/Policy.java ! src/java.base/share/classes/java/security/SecureRandom.java ! src/java.base/share/classes/java/security/SecureRandomSpi.java ! src/java.base/share/classes/java/security/Security.java ! src/java.base/share/classes/java/security/Signature.java ! src/java.base/share/classes/java/security/cert/CertPath.java ! src/java.base/share/classes/java/security/cert/CertPathBuilder.java ! src/java.base/share/classes/java/security/cert/CertPathValidator.java ! src/java.base/share/classes/java/security/cert/CertStore.java ! src/java.base/share/classes/java/security/cert/Certificate.java ! src/java.base/share/classes/java/security/cert/CertificateFactory.java ! src/java.base/share/classes/java/security/cert/CertificateFactorySpi.java ! src/java.base/share/classes/java/security/cert/package-info.java ! src/java.base/share/classes/java/security/package-info.java ! src/java.base/share/classes/java/security/spec/EncodedKeySpec.java ! src/java.base/share/classes/java/security/spec/PKCS8EncodedKeySpec.java ! src/java.base/share/classes/java/security/spec/X509EncodedKeySpec.java ! src/java.base/share/classes/javax/crypto/Cipher.java ! src/java.base/share/classes/javax/crypto/ExemptionMechanism.java ! src/java.base/share/classes/javax/crypto/KeyAgreement.java ! src/java.base/share/classes/javax/crypto/KeyGenerator.java ! src/java.base/share/classes/javax/crypto/Mac.java ! src/java.base/share/classes/javax/crypto/SecretKeyFactory.java ! src/java.base/share/classes/javax/crypto/package-info.java ! src/java.base/share/classes/javax/net/ssl/SSLContext.java ! src/java.base/share/classes/javax/net/ssl/SSLEngine.java ! src/java.base/share/classes/javax/net/ssl/SSLParameters.java ! src/java.base/share/classes/javax/net/ssl/SSLServerSocket.java ! src/java.base/share/classes/javax/net/ssl/SSLServerSocketFactory.java ! src/java.base/share/classes/javax/net/ssl/SSLSocket.java ! src/java.base/share/classes/javax/net/ssl/SSLSocketFactory.java ! src/java.base/share/classes/javax/net/ssl/TrustManagerFactory.java ! src/java.base/share/classes/javax/net/ssl/package-info.java ! src/java.base/share/classes/javax/security/auth/login/Configuration.java ! src/java.base/share/classes/javax/security/auth/login/package-info.java ! src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java Changeset: 81ae2abfb49f Author: prappo Date: 2017-05-10 12:36 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/81ae2abfb49f 8179021: Latest bugfixes to WebSocket/HPACK from the sandbox repo Reviewed-by: dfuchs ! src/java.base/share/classes/module-info.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/HttpRequestImpl.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/PlainTunnelingConnection.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/WebSocket.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/hpack/Decoder.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/hpack/HeaderTable.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/BuilderImpl.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/CooperativeHandler.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/Frame.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/FrameConsumer.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/OpeningHandshake.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/OutgoingMessage.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/Receiver.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/StatusCodes.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/Transmitter.java ! src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/internal/websocket/WebSocketImpl.java ! test/java/net/httpclient/whitebox/Driver.java + test/java/net/httpclient/whitebox/jdk.incubator.httpclient/jdk/incubator/http/RawChannelTest.java Changeset: f12c83ece7ab Author: erikj Date: 2017-05-10 18:27 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/f12c83ece7ab 8179867: JDK9 b167: demos exist in JDK bundles Reviewed-by: ihse ! make/CompileDemos.gmk Changeset: a3af889e7f65 Author: smarks Date: 2017-05-10 15:59 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/a3af889e7f65 8180128: small errors in String javadoc Reviewed-by: lancea ! src/java.base/share/classes/java/lang/String.java Changeset: f895e36ce0e9 Author: msheppar Date: 2017-05-11 15:05 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/f895e36ce0e9 7155591: test/java/net/MulticastSocket/SetOutgoingIf.java fails on macOS Reviewed-by: chegar, bpb ! test/java/net/MulticastSocket/SetOutgoingIf.java Changeset: 4e2a2733855d Author: bpb Date: 2017-05-11 07:43 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/4e2a2733855d 8180085: (ch) java/nio/channels/SocketChannel/VectorIO.java: add debug instrumentation Summary: Add some print statements potentially useful in solving JDK-8177801. Reviewed-by: alanb ! test/java/nio/channels/SocketChannel/VectorIO.java Changeset: c17e1662bad1 Author: jjg Date: 2017-05-11 10:48 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/c17e1662bad1 8179592: Update tables in java.base to be HTML 5-friendly. Reviewed-by: mchung, darcy, martin ! src/java.base/share/classes/java/io/DataInput.java ! src/java.base/share/classes/java/io/RandomAccessFile.java ! src/java.base/share/classes/java/io/SerializablePermission.java ! src/java.base/share/classes/java/lang/Character.java ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/Double.java ! src/java.base/share/classes/java/lang/Float.java ! src/java.base/share/classes/java/lang/RuntimePermission.java ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java ! src/java.base/share/classes/java/lang/invoke/MethodHandleInfo.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/java/lang/reflect/AnnotatedElement.java ! src/java.base/share/classes/java/lang/reflect/ReflectPermission.java ! src/java.base/share/classes/java/math/BigDecimal.java ! src/java.base/share/classes/java/math/RoundingMode.java ! src/java.base/share/classes/java/net/Inet4Address.java ! src/java.base/share/classes/java/net/Inet6Address.java ! src/java.base/share/classes/java/net/InetAddress.java ! src/java.base/share/classes/java/net/NetPermission.java ! src/java.base/share/classes/java/net/URI.java ! src/java.base/share/classes/java/net/URLConnection.java ! src/java.base/share/classes/java/net/URLPermission.java ! src/java.base/share/classes/java/nio/channels/AsynchronousChannelGroup.java ! src/java.base/share/classes/java/nio/channels/AsynchronousFileChannel.java ! src/java.base/share/classes/java/nio/channels/AsynchronousServerSocketChannel.java ! src/java.base/share/classes/java/nio/channels/AsynchronousSocketChannel.java ! src/java.base/share/classes/java/nio/channels/DatagramChannel.java ! src/java.base/share/classes/java/nio/channels/FileChannel.java ! src/java.base/share/classes/java/nio/channels/ServerSocketChannel.java ! src/java.base/share/classes/java/nio/channels/SocketChannel.java ! src/java.base/share/classes/java/nio/charset/Charset.java ! src/java.base/share/classes/java/nio/file/FileSystem.java ! src/java.base/share/classes/java/nio/file/Files.java ! src/java.base/share/classes/java/nio/file/LinkPermission.java ! src/java.base/share/classes/java/nio/file/attribute/AclFileAttributeView.java ! src/java.base/share/classes/java/nio/file/attribute/BasicFileAttributeView.java ! src/java.base/share/classes/java/nio/file/attribute/DosFileAttributeView.java ! src/java.base/share/classes/java/nio/file/attribute/PosixFileAttributeView.java ! src/java.base/share/classes/java/nio/file/spi/FileSystemProvider.java ! src/java.base/share/classes/java/security/DrbgParameters.java ! src/java.base/share/classes/java/security/Provider.java ! src/java.base/share/classes/java/security/SecurityPermission.java ! src/java.base/share/classes/java/security/cert/X509Extension.java ! src/java.base/share/classes/java/text/DecimalFormat.java ! src/java.base/share/classes/java/text/MessageFormat.java ! src/java.base/share/classes/java/text/SimpleDateFormat.java ! src/java.base/share/classes/java/time/chrono/HijrahChronology.java ! src/java.base/share/classes/java/time/chrono/IsoEra.java ! src/java.base/share/classes/java/time/chrono/MinguoEra.java ! src/java.base/share/classes/java/time/chrono/ThaiBuddhistEra.java ! src/java.base/share/classes/java/time/format/DateTimeFormatter.java ! src/java.base/share/classes/java/time/temporal/IsoFields.java ! src/java.base/share/classes/java/time/temporal/WeekFields.java ! src/java.base/share/classes/java/util/Deque.java ! src/java.base/share/classes/java/util/Formatter.java ! src/java.base/share/classes/java/util/GregorianCalendar.java ! src/java.base/share/classes/java/util/Locale.java ! src/java.base/share/classes/java/util/Queue.java ! src/java.base/share/classes/java/util/ResourceBundle.java ! src/java.base/share/classes/java/util/concurrent/BlockingDeque.java ! src/java.base/share/classes/java/util/concurrent/BlockingQueue.java ! src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java ! src/java.base/share/classes/java/util/regex/Pattern.java ! src/java.base/share/classes/java/util/spi/CalendarNameProvider.java ! src/java.base/share/classes/java/util/stream/StreamOpFlag.java ! src/java.base/share/classes/javax/net/ssl/SSLPermission.java Changeset: 5724641773d5 Author: lana Date: 2017-05-11 18:10 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/5724641773d5 Merge Changeset: 93594ad5db2a Author: jjg Date: 2017-05-11 18:04 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/93594ad5db2a 8180256: Fix HTML 5 issues in java.sql and java.sql.rowset modules Reviewed-by: lancea ! src/java.sql.rowset/share/classes/javax/sql/rowset/package.html ! src/java.sql.rowset/share/classes/javax/sql/rowset/spi/package.html ! src/java.sql/share/classes/java/sql/SQLPermission.java ! src/java.sql/share/classes/java/sql/Statement.java Changeset: bb4cdc198dc0 Author: dl Date: 2017-05-11 19:34 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/bb4cdc198dc0 8179515: Class java.util.concurrent.ThreadLocalRandom fails to Initialize when using SecurityManager Summary: Break connection to security framework by using VM.getSavedProperty to read the proeprty. Reviewed-by: dholmes, martin, redestad, psandoz, mchung ! src/java.base/share/classes/java/util/concurrent/ThreadLocalRandom.java Changeset: 7fb801c87a25 Author: dholmes Date: 2017-05-12 02:06 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7fb801c87a25 Merge Changeset: 0111d77e0170 Author: michaelm Date: 2017-05-12 17:38 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/0111d77e0170 8180303: Remove technote doc link from ProxySelector/B8035158.java test Reviewed-by: chegar ! test/java/net/ProxySelector/B8035158.java Changeset: 9364a094ce56 Author: ksrini Date: 2017-05-12 09:42 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/9364a094ce56 8179631: Fix Html5 errors in java.management, jdk.management, jdk.jdi and jdk.attach Reviewed-by: mchung ! src/java.management/share/classes/java/lang/management/ManagementPermission.java ! src/java.management/share/classes/java/lang/management/ThreadInfo.java ! src/java.management/share/classes/javax/management/Descriptor.java ! src/java.management/share/classes/javax/management/MXBean.java ! src/java.management/share/classes/javax/management/modelmbean/ModelMBeanAttributeInfo.java ! src/java.management/share/classes/javax/management/modelmbean/ModelMBeanConstructorInfo.java ! src/java.management/share/classes/javax/management/modelmbean/ModelMBeanInfo.java ! src/java.management/share/classes/javax/management/modelmbean/ModelMBeanNotificationInfo.java ! src/java.management/share/classes/javax/management/modelmbean/ModelMBeanOperationInfo.java ! src/jdk.attach/share/classes/com/sun/tools/attach/AttachPermission.java ! src/jdk.jdi/share/classes/com/sun/jdi/JDIPermission.java ! src/jdk.jdi/share/classes/com/sun/jdi/doc-files/signature.html ! src/jdk.management/share/classes/com/sun/management/DiagnosticCommandMBean.java ! src/jdk.management/share/classes/com/sun/management/GarbageCollectionNotificationInfo.java ! src/jdk.management/share/classes/com/sun/management/GcInfo.java ! src/jdk.management/share/classes/com/sun/management/VMOption.java Changeset: 1beb24516aa1 Author: bchristi Date: 2017-05-12 10:11 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/1beb24516aa1 8177328: java/lang/ClassLoader/securityManager/ClassLoaderTest.java times out with -Xcomp Summary: remove unneeded automatic module tests; refactor into multiple @runs Reviewed-by: mchung ! test/java/lang/ClassLoader/securityManager/ClassLoaderTest.java Changeset: fb6138cce508 Author: ksrini Date: 2017-05-12 10:26 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/fb6138cce508 8179697: Fix Html5 errors in java.naming, java.logging, jdk.httpserver, jdk.net, jdk.sctp Reviewed-by: dfuchs ! src/java.logging/share/classes/java/util/logging/LogManager.java ! src/java.logging/share/classes/java/util/logging/SimpleFormatter.java ! src/java.naming/share/classes/javax/naming/CompositeName.java ! src/java.naming/share/classes/javax/naming/Context.java ! src/java.naming/share/classes/javax/naming/InitialContext.java ! src/java.naming/share/classes/javax/naming/directory/package.html ! src/java.naming/share/classes/javax/naming/event/package.html ! src/java.naming/share/classes/javax/naming/ldap/package.html ! src/java.naming/share/classes/javax/naming/package.html ! src/java.naming/share/classes/javax/naming/spi/ObjectFactory.java ! src/java.naming/share/classes/javax/naming/spi/package.html ! src/jdk.httpserver/share/classes/com/sun/net/httpserver/HttpServer.java ! src/jdk.httpserver/share/classes/com/sun/net/httpserver/HttpsConfigurator.java ! src/jdk.net/share/classes/jdk/net/NetworkPermission.java ! src/jdk.sctp/share/classes/com/sun/nio/sctp/SctpChannel.java ! src/jdk.sctp/share/classes/com/sun/nio/sctp/SctpMultiChannel.java ! src/jdk.sctp/share/classes/com/sun/nio/sctp/SctpServerChannel.java Changeset: 65e09a42b587 Author: jjg Date: 2017-04-28 15:41 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/65e09a42b587 8179460: Fix unnecessary uses of {@docRoot} in serviceability APIs Reviewed-by: mchung ! src/jdk.jconsole/share/classes/com/sun/tools/jconsole/JConsolePlugin.java ! src/jdk.jdi/share/classes/com/sun/jdi/ThreadGroupReference.java ! src/jdk.management/share/classes/com/sun/management/DiagnosticCommandMBean.java ! src/jdk.management/share/classes/com/sun/management/package-info.java Changeset: 29e8713fdd03 Author: jjg Date: 2017-04-28 15:43 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/29e8713fdd03 Merge Changeset: e2b414957632 Author: jjg Date: 2017-05-12 10:43 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e2b414957632 Merge ! src/jdk.management/share/classes/com/sun/management/DiagnosticCommandMBean.java - test/demo/jvmti/Context.java - test/demo/jvmti/DemoRun.java - test/demo/jvmti/HeapUser.java - test/demo/jvmti/Hello.java - test/demo/jvmti/compiledMethodLoad/CompiledMethodLoadTest.java - test/demo/jvmti/gctest/BigHello.java - test/demo/jvmti/gctest/Gctest.java - test/demo/jvmti/heapTracker/HeapTrackerTest.java - test/demo/jvmti/heapViewer/HeapViewerTest.java - test/demo/jvmti/minst/MinstExample.java - test/demo/jvmti/minst/MinstTest.java - test/demo/jvmti/versionCheck/FailsWhenJvmtiVersionDiffers.java - test/demo/jvmti/waiters/WaitersTest.java - test/sample/TEST.properties - test/sample/chatserver/ChatTest.java - test/sample/mergesort/MergeSortTest.java - test/tools/launcher/modules/listmods/src/java.transaction/javax/transaction/atomic/Atomic.java Changeset: e8ee0b0489eb Author: dfuchs Date: 2017-05-12 19:06 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e8ee0b0489eb 8180176: Broken javadoc links in java.logging and java.naming Summary: Replace relative hrefs with {@extLink } taglets. Reviewed-by: chegar, mchung, vtewari ! src/java.logging/share/classes/java/util/logging/package.html ! src/java.naming/share/classes/javax/naming/directory/package.html ! src/java.naming/share/classes/javax/naming/event/package.html ! src/java.naming/share/classes/javax/naming/ldap/package.html ! src/java.naming/share/classes/javax/naming/package.html ! src/java.naming/share/classes/javax/naming/spi/package.html Changeset: 002321560796 Author: lancea Date: 2017-05-12 14:35 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/002321560796 8180309: Minor update to javax.sql.rowset package.html Reviewed-by: mchung ! src/java.sql.rowset/share/classes/javax/sql/rowset/package.html Changeset: a046521803b2 Author: rriggs Date: 2017-05-12 15:15 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/a046521803b2 8180082: Broken javadoc links Reviewed-by: mchung, bpb, chegar, ihse ! src/java.base/share/classes/java/io/ObjectInputStream.java ! src/java.base/share/classes/java/io/ObjectOutputStream.java ! src/java.base/share/classes/java/io/ObjectStreamClass.java ! src/java.base/share/classes/java/io/package-info.java ! src/java.base/share/classes/java/lang/CharSequence.java ! src/java.base/share/classes/java/lang/ModuleLayer.java ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/security/Key.java ! src/java.base/share/classes/java/security/KeyRep.java ! src/java.base/share/specs/serialization/output.md ! src/java.management/share/classes/java/lang/management/package.html ! src/jdk.management/share/classes/com/sun/management/package-info.java Changeset: 9fc68c99204d Author: mchung Date: 2017-05-12 13:29 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/9fc68c99204d 8180208: Provide a new docs bundle page Reviewed-by: ihse, jjg ! make/CompileModuleTools.gmk ! make/CompileTools.gmk ! make/ModuleTools.gmk + make/src/classes/build/tools/docs/GenDocsBundlePage.java + make/src/classes/build/tools/docs/docs-bundle-page.html + make/src/classes/build/tools/docs/docs-module-groups.properties Changeset: c5b0465f06d0 Author: psandoz Date: 2017-05-12 13:43 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/c5b0465f06d0 8180075: Javadoc of MethodHandles.Lookup::bind should note the difference from MethodHandle::bindTo Reviewed-by: psandoz Contributed-by: ron.pressler at oracle.com ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java Changeset: 661765f8b131 Author: smarks Date: 2017-05-12 14:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/661765f8b131 8180137: fix broken link in java.lang.Iterable Reviewed-by: rriggs ! src/java.base/share/classes/java/lang/Iterable.java Changeset: ef9954f6896b Author: rriggs Date: 2017-05-16 09:42 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/ef9954f6896b 8180319: Update Serialization spec to omit obsolete serialver -show and change history Reviewed-by: chegar - src/java.base/share/specs/serialization/changelog.md ! src/java.base/share/specs/serialization/class.md - src/java.base/share/specs/serialization/images/class.gif ! src/java.base/share/specs/serialization/index.md ! src/java.base/share/specs/serialization/security.md ! src/java.base/share/specs/serialization/serial-arch.md Changeset: 1b3ea9753746 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/1b3ea9753746 Added tag jdk-9+170 for changeset ef9954f6896b ! .hgtags Changeset: 8ad82afd0b52 Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/8ad82afd0b52 Merge ! .hgtags ! make/CompileDemos.gmk ! make/CompileModuleTools.gmk ! make/ModuleTools.gmk ! src/java.base/share/classes/java/io/ObjectInputStream.java ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/ModuleLayer.java ! src/java.base/share/classes/java/lang/RuntimePermission.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/invoke/MethodHandles.java ! src/java.base/share/classes/java/nio/charset/Charset.java ! src/java.base/share/classes/java/nio/file/Files.java ! src/java.base/share/classes/java/security/Provider.java ! src/java.base/share/classes/java/util/ResourceBundle.java ! src/java.base/share/classes/module-info.java ! src/java.logging/share/classes/java/util/logging/LogManager.java ! src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java From mandy.chung at oracle.com Thu May 18 21:21:15 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:21:15 +0000 Subject: hg: jigsaw/jake/langtools: 7 new changesets Message-ID: <201705182121.v4ILLFaE017688@aojmv0008.oracle.com> Changeset: ee84b7d44339 Author: jjg Date: 2017-05-09 17:20 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/ee84b7d44339 8179479: Add new styles to enable HTML 5 tables Reviewed-by: bpatel ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/stylesheet.css Changeset: d5c5150ad2d9 Author: lana Date: 2017-05-11 18:11 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/d5c5150ad2d9 Merge Changeset: 6ffca66728a7 Author: ksrini Date: 2017-05-11 15:12 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/6ffca66728a7 8179632: Fix the old doclet documentation Reviewed-by: jjg ! src/jdk.javadoc/share/classes/com/sun/javadoc/package-info.java ! src/jdk.javadoc/share/classes/com/sun/tools/doclets/package-info.java + src/jdk.javadoc/share/classes/com/sun/tools/doclets/standard/package-info.java Changeset: 77a2d6c1f321 Author: jlahoda Date: 2017-05-12 06:42 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/77a2d6c1f321 8178152: Handling of incubating modules, the jdk.unsupported module and --add-exports with --release Summary: --release 9 should only allow documented modules; --add-exports/--add-reads/--patch-module should not be allowed on system modules when --release 9 is used. Reviewed-by: jjg, erikj, ihse ! make/gendata/Gendata-jdk.compiler.gmk + make/src/classes/build/tools/symbolgenerator/TransitiveDependencies.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Modules.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/Arguments.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/platform/JDKPlatformProvider.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties + test/tools/javac/diags/examples/AddExportsWithRelease.java + test/tools/javac/diags/examples/AddReadsWithRelease.java + test/tools/javac/diags/examples/PatchModuleWithRelease/PatchModuleWithRelease.java + test/tools/javac/diags/examples/PatchModuleWithRelease/patchmodule/java.base/java/lang/Test.java ! test/tools/javac/options/release/ReleaseOptionClashes.java + test/tools/javac/options/release/ReleaseOptionUnsupported.java Changeset: 18355c879c69 Author: bpatel Date: 2017-05-12 18:05 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/18355c879c69 8178043: Support grouping modules in unified javadoc Reviewed-by: jjg, ksrini ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractModuleIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexFrameWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModulePackageIndexFrameWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PackageIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/Configuration.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Group.java ! test/jdk/javadoc/doclet/testModules/TestModules.java Changeset: aae59039c1f5 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/aae59039c1f5 Added tag jdk-9+170 for changeset 18355c879c69 ! .hgtags Changeset: a543a82a77b7 Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/a543a82a77b7 Merge ! .hgtags ! make/gendata/Gendata-jdk.compiler.gmk ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Modules.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/Arguments.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractModuleIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModuleIndexFrameWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/ModulePackageIndexFrameWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PackageIndexWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/Configuration.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/stylesheet.css ! test/jdk/javadoc/doclet/testModules/TestModules.java ! test/tools/javac/options/release/ReleaseOptionClashes.java From mandy.chung at oracle.com Thu May 18 21:21:18 2017 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 18 May 2017 21:21:18 +0000 Subject: hg: jigsaw/jake/nashorn: 3 new changesets Message-ID: <201705182121.v4ILLIKF017738@aojmv0008.oracle.com> Changeset: 550bfc15779f Author: sdama Date: 2017-05-12 12:12 +0530 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/550bfc15779f 8179891: JavaDoc for for..in is incorrect Summary: Fixed javadoc typo error Reviewed-by: sundar, hannesw, jlaskey Contributed-by: srinivas.dama at oracle.com ! src/jdk.scripting.nashorn/share/classes/jdk/nashorn/api/tree/ForOfLoopTree.java Changeset: fc416270a776 Author: lana Date: 2017-05-18 14:54 +0000 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/fc416270a776 Added tag jdk-9+170 for changeset 550bfc15779f ! .hgtags Changeset: ff58cb77610f Author: mchung Date: 2017-05-18 14:18 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/ff58cb77610f Merge ! .hgtags From nipa at codefx.org Thu May 18 23:17:17 2017 From: nipa at codefx.org (Nicolai Parlog) Date: Fri, 19 May 2017 01:17:17 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> Hi! I like the flag per se and think a "quiet" argument would be helpful. I also understand that there is pressure to improve the spec and that many observers will see this change as progress[1]. But I think making the lenient option the default is a bad decision, needlessly prolonging a proper clean-up of the ecosystem without any practical benefit! Java's stewards have been warning against using internal APIs for 20 years. Jigsaw announced to make them inaccessible for nigh two years now. Java 8 will be supported until at least July 2018 and even after that all that is needed to get around this specific Java 9 compatibility problem is to add `--permit-illegal-access`. All that is not enough, though? Even adding that single flag is too hard? If spending an hour or two reading up on JPMS and then adding that flag is too much to ask, then how can one expect the same people to clean up their code? With illegal access being permitted by default much fewer developers will be aware of the problem and much less pressure will be put on library and framework maintainers as well as on project management to invest into paying back this particular form of technical debt. So we get much less momentum to make the necessary changes in exchange for... not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the Java community! so long ... Nicolai [1](where in fact it only creates the appearance thereof) [2] https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html On 18.05.2017 16:48, mark.reinhold at oracle.com wrote: > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > [2] http://openjdk.java.net/jeps/260 > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From john.r.rose at oracle.com Fri May 19 04:24:29 2017 From: john.r.rose at oracle.com (John Rose) Date: Thu, 18 May 2017 21:24:29 -0700 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: <287E4BFB-3616-4530-8F2C-4C478E9E3DA4@oracle.com> On May 18, 2017, at 1:59 AM, Stephan Herrmann wrote: > > In all posts I could not find a real reason against escaping, > aside from aesthetics. I don't see this as sufficient motivation > for a less-then-perfect solution. So, by disregarding esthetics... > > Clarity: > I'm still not completely following your explanations, partly because > of the jargon you are using. I'll leave it to Alex to decide if he > likes the idea that JLS would have to explain terms like dotted > production. > > Compare this to just adding a few more rules to the grammar, > where no hand-waving is needed for an explanation. > No, I did not say that escaping is a pervasive change. > I never said that the grammar for ordinary compilation units > should be changed. > If you like we only need to extend one rule for the scope of > modular compilation units: Identifier. It can't get simpler. > > > Completeness: > I understand you as saying, module names cannot start with > "transitive". Mind you, that every modifier that will be added > to the grammar for modules in the future will cause conflicts for > names that are now legal, and you won't have a means to resolve this. > > By contrast, we can use the escaping approach even to solve one > more problem that has been briefly touched on this list before: > > Automatic modules suffer from the fact that some artifact names may > have Java keywords in their name, which means that these artifacts > simply cannot be used as automatic modules, right? > Why not apply escaping also here? *Any* dot-separated sequence > of words could be used as module name, as long as module references > have a means to escape any keywords in that sequence. > > > Suitability for implementation: > As said, your proposal resolves one problem, but still IDE > functionality suffers from restricted keywords, because scanning > and parsing need more context information than normal. ?we obtain the freedom for IDEs to disregard abnormal amounts of context, saving uncounted machine cycles, > - Recovery after a syntax error will regress. ?and we make life easier for all ten writers of error recovery functions, > - Scanning arbitrary regions of code is not possible. ?we unleash the power of an army of grad students to study bidirectional parsing of module files, > Remember: > In an IDE code with syntax errors is the norm, not an exception, > as the IDE provides functionality to work on incomplete code. ?and ease the burdens of the thousands who must spend their time looking at syntax errors for their broken module files. Nope, not for me. Give me esthetics, please. Really. ? John From chris at hazelcast.com Fri May 19 04:42:39 2017 From: chris at hazelcast.com (Christoph Engelbert) Date: Fri, 19 May 2017 06:42:39 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> Message-ID: <492EDA7C-49D3-4694-8C70-EF3E71F74EAA@hazelcast.com> Hey, I also think the new styled parameter is great and I would appreciate a fully ?quiet? mode for our customers. I don?t mandate it to be the default but I think to make it the default for (and only for) Java 9 sounds meaningful. The reason is simple: It is not just us (a few people, a few vendors) to set this parameter but it will be tons of customers or users, running the actual production workloads, to add it to their command line. I guess most of us know the burden of making a change to the command line at some bigger companies, such as banks. On the other side, as a dev I want it to print as much warnings as possible, when I?m working on fixing the actual ?misbehaving? code fragment. In general I?m ok with making users add it to their command line but would like to see something like quiet as the default for Java 9. PS: the `?illegal-access=permit` as default works for me though :) Thanks for the revised proposal, good stuff! Cheers, Chris > On 19. May 2017, at 01:17, Nicolai Parlog wrote: > > Hi! > > I like the flag per se and think a "quiet" argument would be helpful. I > also understand that there is pressure to improve the spec and that many > observers will see this change as progress[1]. > > But I think making the lenient option the default is a bad decision, > needlessly prolonging a proper clean-up of the ecosystem without any > practical benefit! > > Java's stewards have been warning against using internal APIs for 20 > years. Jigsaw announced to make them inaccessible for nigh two years > now. Java 8 will be supported until at least July 2018 and even after > that all that is needed to get around this specific Java 9 compatibility > problem is to add `--permit-illegal-access`. > > All that is not enough, though? Even adding that single flag is too > hard? If spending an hour or two reading up on JPMS and then adding that > flag is too much to ask, then how can one expect the same people to > clean up their code? > > With illegal access being permitted by default much fewer developers > will be aware of the problem and much less pressure will be put on > library and framework maintainers as well as on project management to > invest into paying back this particular form of technical debt. So we > get much less momentum to make the necessary changes in exchange for... > not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the > Java community! > > so long ... Nicolai > > > [1](where in fact it only creates the appearance thereof) > [2] > https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html > > > > On 18.05.2017 16:48, mark.reinhold at oracle.com wrote: >> Over time, as we've gotten closer and closer to the JDK 9 GA date, more >> and more developers have begun paying attention the actual changes in >> this release. The strong encapsulation of JDK-internal APIs has, in >> particular, triggered many worried expressions of concern that code that >> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance >> warning of this change was given in JDK 8. >> >> To help the entire ecosystem migrate to the modular Java platform at a >> more relaxed pace I hereby propose to allow illegal reflective access >> from code on the class path by default in JDK 9, and to disallow it in >> a future release. >> >> In short, the existing "big kill switch" of the `--permit-illegal-access` >> option [1] will become the default behavior of the JDK 9 run-time system, >> though without as many warnings. The current behavior of JDK 9, in which >> illegal reflective-access operations from code on the class path are not >> permitted, will become the default in a future release. Nothing will >> change at compile time. >> >> In detail, the recently-introduced `--permit-illegal-access` option will >> be replaced by a more-general option, `--illegal-access`. This option >> will take a single keyword parameter, as follows: >> >> `--illegal-access=permit` >> >> This will be the default mode for JDK 9. It opens every package in >> every explicit module to code in all unnamed modules, i.e., code on >> the class path, just as `--permit-illegal-access` does today. >> >> The first illegal reflective-access operation causes a warning to be >> issued, as with `--permit-illegal-access`, but no warnings are issued >> after that point. This single warning will describe how to enable >> further warnings. >> >> `--illegal-access=warn` >> >> This causes a warning message to be issued for each illegal >> reflective-access operation. This is equivalent to the current >> `--permit-illegal-access` option. >> >> `--illegal-access=debug` >> >> This causes both a warning message and a stack trace to be shown >> for each illegal reflective-access operation. This is equivalent >> to combining today's `--permit-illegal-access` option with >> `-Dsun.reflect.debugModuleAccessChecks`. >> >> `--illegal-access=deny` >> >> This disables all illegal reflective-access operations except for >> those enabled by other command-line options, such as `--add-opens`. >> This will become the default mode in a future release. >> >> Notes: >> >> - The proposed default mode enables the run-time system to issue a >> warning message, possibly at some time long after startup, without >> having been explicitly requested to do so. This may be a surprise >> in production environments, since it's extremely unusual for the >> run-time system to issue any warning messages at all. If the default >> mode permits illegal reflective access, however, then it's essential >> to make that known so that people aren't surprised when this is no >> longer the default mode in a future release. >> >> - Warning messages in any mode can be avoided, as before, by the >> judicious use of the `--add-exports` and `--add-opens` options. >> >> - This proposal will, if adopted, require adjustments to JEP 260, >> "Encapsulate Most Internal APIs" [2]. APIs that are internal to the >> JDK will still be strongly encapsulated from the standpoint of code >> in modules, whether those modules are automatic or explicit, but they >> will not appear to be encapsulated at run time from the standpoint of >> code on the class path. >> >> - When `deny` becomes the default mode then I expect `permit` to remain >> supported for at least one release, so that developers can continue >> to migrate their code. The `permit`, `warn`, and `debug` modes will, >> over time, be removed, as will the `--illegal-access` option itself. >> (For launch-script compatibility the unsupported modes will most >> likely just be ignored, after issuing a warning to that effect.) >> >> - This change will not magically solve every JDK 9 adoption problem. >> The concrete types of the built-in class loaders are still different, >> `rt.jar` is still gone, the layout of a system image is still not the >> same, and the version string still has a new format. >> >> Comments? >> >> - Mark >> >> >> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html >> [2] http://openjdk.java.net/jeps/260 >> > > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > https://www.sitepoint.com/java > high-quality Java/JVM content > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx From weijun.wang at oracle.com Fri May 19 05:10:01 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 19 May 2017 13:10:01 +0800 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <6ce59987-31ef-8b9d-d5bd-9ffe4445faa5@oracle.com> I don't like this default value, but I also know some people wanting it. Is it possible to provide the default value in a file inside the conf directory and also make it overwrite-able on the command line? Maybe RedHat Linux can make it "permit" out-of-box and other vendors can choose different values. Thanks Max On 05/18/2017 10:48 PM, mark.reinhold at oracle.com wrote: > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > [2] http://openjdk.java.net/jeps/260 > From amaembo at gmail.com Fri May 19 05:18:02 2017 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 19 May 2017 12:18:02 +0700 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <492EDA7C-49D3-4694-8C70-EF3E71F74EAA@hazelcast.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <492EDA7C-49D3-4694-8C70-EF3E71F74EAA@hazelcast.com> Message-ID: Regarding proposed "quiet" option: there's existing non-standard -XX:-PrintWarnings JVM option. I wonder whether it will work to suppress the illegal access warning as well. If yes, then no additional "quite" mode is necessary. With best regards, Tagir Valeev. On Fri, May 19, 2017 at 11:42 AM, Christoph Engelbert wrote: > Hey, > > I also think the new styled parameter is great and I would appreciate a > fully > ?quiet? mode for our customers. I don?t mandate it to be the default but I > think > to make it the default for (and only for) Java 9 sounds meaningful. > > The reason is simple: It is not just us (a few people, a few vendors) to > set > this parameter but it will be tons of customers or users, running the > actual > production workloads, to add it to their command line. I guess most of us > know the burden of making a change to the command line at some bigger > companies, such as banks. > > On the other side, as a dev I want it to print as much warnings as > possible, > when I?m working on fixing the actual ?misbehaving? code fragment. > > In general I?m ok with making users add it to their command line but would > like to see something like quiet as the default for Java 9. > > PS: the `?illegal-access=permit` as default works for me though :) > > Thanks for the revised proposal, good stuff! > > Cheers, > Chris > > > > On 19. May 2017, at 01:17, Nicolai Parlog wrote: > > > > Hi! > > > > I like the flag per se and think a "quiet" argument would be helpful. I > > also understand that there is pressure to improve the spec and that many > > observers will see this change as progress[1]. > > > > But I think making the lenient option the default is a bad decision, > > needlessly prolonging a proper clean-up of the ecosystem without any > > practical benefit! > > > > Java's stewards have been warning against using internal APIs for 20 > > years. Jigsaw announced to make them inaccessible for nigh two years > > now. Java 8 will be supported until at least July 2018 and even after > > that all that is needed to get around this specific Java 9 compatibility > > problem is to add `--permit-illegal-access`. > > > > All that is not enough, though? Even adding that single flag is too > > hard? If spending an hour or two reading up on JPMS and then adding that > > flag is too much to ask, then how can one expect the same people to > > clean up their code? > > > > With illegal access being permitted by default much fewer developers > > will be aware of the problem and much less pressure will be put on > > library and framework maintainers as well as on project management to > > invest into paying back this particular form of technical debt. So we > > get much less momentum to make the necessary changes in exchange for... > > not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the > > Java community! > > > > so long ... Nicolai > > > > > > [1](where in fact it only creates the appearance thereof) > > [2] > > https://krautblog-ulrich.blogspot.de/2012/07/word-of- > month-armutszeugnis.html > > > > > > > > On 18.05.2017 16:48, mark.reinhold at oracle.com wrote: > >> Over time, as we've gotten closer and closer to the JDK 9 GA date, more > >> and more developers have begun paying attention the actual changes in > >> this release. The strong encapsulation of JDK-internal APIs has, in > >> particular, triggered many worried expressions of concern that code that > >> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > >> warning of this change was given in JDK 8. > >> > >> To help the entire ecosystem migrate to the modular Java platform at a > >> more relaxed pace I hereby propose to allow illegal reflective access > >> from code on the class path by default in JDK 9, and to disallow it in > >> a future release. > >> > >> In short, the existing "big kill switch" of the > `--permit-illegal-access` > >> option [1] will become the default behavior of the JDK 9 run-time > system, > >> though without as many warnings. The current behavior of JDK 9, in > which > >> illegal reflective-access operations from code on the class path are not > >> permitted, will become the default in a future release. Nothing will > >> change at compile time. > >> > >> In detail, the recently-introduced `--permit-illegal-access` option will > >> be replaced by a more-general option, `--illegal-access`. This option > >> will take a single keyword parameter, as follows: > >> > >> `--illegal-access=permit` > >> > >> This will be the default mode for JDK 9. It opens every package in > >> every explicit module to code in all unnamed modules, i.e., code on > >> the class path, just as `--permit-illegal-access` does today. > >> > >> The first illegal reflective-access operation causes a warning to be > >> issued, as with `--permit-illegal-access`, but no warnings are issued > >> after that point. This single warning will describe how to enable > >> further warnings. > >> > >> `--illegal-access=warn` > >> > >> This causes a warning message to be issued for each illegal > >> reflective-access operation. This is equivalent to the current > >> `--permit-illegal-access` option. > >> > >> `--illegal-access=debug` > >> > >> This causes both a warning message and a stack trace to be shown > >> for each illegal reflective-access operation. This is equivalent > >> to combining today's `--permit-illegal-access` option with > >> `-Dsun.reflect.debugModuleAccessChecks`. > >> > >> `--illegal-access=deny` > >> > >> This disables all illegal reflective-access operations except for > >> those enabled by other command-line options, such as `--add-opens`. > >> This will become the default mode in a future release. > >> > >> Notes: > >> > >> - The proposed default mode enables the run-time system to issue a > >> warning message, possibly at some time long after startup, without > >> having been explicitly requested to do so. This may be a surprise > >> in production environments, since it's extremely unusual for the > >> run-time system to issue any warning messages at all. If the default > >> mode permits illegal reflective access, however, then it's essential > >> to make that known so that people aren't surprised when this is no > >> longer the default mode in a future release. > >> > >> - Warning messages in any mode can be avoided, as before, by the > >> judicious use of the `--add-exports` and `--add-opens` options. > >> > >> - This proposal will, if adopted, require adjustments to JEP 260, > >> "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > >> JDK will still be strongly encapsulated from the standpoint of code > >> in modules, whether those modules are automatic or explicit, but they > >> will not appear to be encapsulated at run time from the standpoint of > >> code on the class path. > >> > >> - When `deny` becomes the default mode then I expect `permit` to remain > >> supported for at least one release, so that developers can continue > >> to migrate their code. The `permit`, `warn`, and `debug` modes will, > >> over time, be removed, as will the `--illegal-access` option itself. > >> (For launch-script compatibility the unsupported modes will most > >> likely just be ignored, after issuing a warning to that effect.) > >> > >> - This change will not magically solve every JDK 9 adoption problem. > >> The concrete types of the built-in class loaders are still different, > >> `rt.jar` is still gone, the layout of a system image is still not the > >> same, and the version string still has a new format. > >> > >> Comments? > >> > >> - Mark > >> > >> > >> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017- > March/011763.html > >> [2] http://openjdk.java.net/jeps/260 > >> > > > > -- > > > > PGP Key: > > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > > > Web: > > http://codefx.org > > a blog about software development > > https://www.sitepoint.com/java > > high-quality Java/JVM content > > http://do-foss.de > > Free and Open Source Software for the City of Dortmund > > > > Twitter: > > https://twitter.com/nipafx > > From chris at hazelcast.com Fri May 19 05:41:07 2017 From: chris at hazelcast.com (Christoph Engelbert) Date: Fri, 19 May 2017 07:41:07 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <492EDA7C-49D3-4694-8C70-EF3E71F74EAA@hazelcast.com> Message-ID: There was a bug report which is closed, but that was, obviously, before the changed proposal. I think it would be subject to reconsider: https://bugs.openjdk.java.net/browse/JDK-8177916 Chris > On 19. May 2017, at 07:18, Tagir Valeev wrote: > > Regarding proposed "quiet" option: there's existing non-standard -XX:-PrintWarnings JVM option. I wonder whether it will work to suppress the illegal access warning as well. If yes, then no additional "quite" mode is necessary. > > With best regards, > Tagir Valeev. > > On Fri, May 19, 2017 at 11:42 AM, Christoph Engelbert > wrote: > Hey, > > I also think the new styled parameter is great and I would appreciate a fully > ?quiet? mode for our customers. I don?t mandate it to be the default but I think > to make it the default for (and only for) Java 9 sounds meaningful. > > The reason is simple: It is not just us (a few people, a few vendors) to set > this parameter but it will be tons of customers or users, running the actual > production workloads, to add it to their command line. I guess most of us > know the burden of making a change to the command line at some bigger > companies, such as banks. > > On the other side, as a dev I want it to print as much warnings as possible, > when I?m working on fixing the actual ?misbehaving? code fragment. > > In general I?m ok with making users add it to their command line but would > like to see something like quiet as the default for Java 9. > > PS: the `?illegal-access=permit` as default works for me though :) > > Thanks for the revised proposal, good stuff! > > Cheers, > Chris > > > > On 19. May 2017, at 01:17, Nicolai Parlog > wrote: > > > > Hi! > > > > I like the flag per se and think a "quiet" argument would be helpful. I > > also understand that there is pressure to improve the spec and that many > > observers will see this change as progress[1]. > > > > But I think making the lenient option the default is a bad decision, > > needlessly prolonging a proper clean-up of the ecosystem without any > > practical benefit! > > > > Java's stewards have been warning against using internal APIs for 20 > > years. Jigsaw announced to make them inaccessible for nigh two years > > now. Java 8 will be supported until at least July 2018 and even after > > that all that is needed to get around this specific Java 9 compatibility > > problem is to add `--permit-illegal-access`. > > > > All that is not enough, though? Even adding that single flag is too > > hard? If spending an hour or two reading up on JPMS and then adding that > > flag is too much to ask, then how can one expect the same people to > > clean up their code? > > > > With illegal access being permitted by default much fewer developers > > will be aware of the problem and much less pressure will be put on > > library and framework maintainers as well as on project management to > > invest into paying back this particular form of technical debt. So we > > get much less momentum to make the necessary changes in exchange for... > > not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the > > Java community! > > > > so long ... Nicolai > > > > > > [1](where in fact it only creates the appearance thereof) > > [2] > > https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html > > > > > > > > On 18.05.2017 16:48, mark.reinhold at oracle.com wrote: > >> Over time, as we've gotten closer and closer to the JDK 9 GA date, more > >> and more developers have begun paying attention the actual changes in > >> this release. The strong encapsulation of JDK-internal APIs has, in > >> particular, triggered many worried expressions of concern that code that > >> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > >> warning of this change was given in JDK 8. > >> > >> To help the entire ecosystem migrate to the modular Java platform at a > >> more relaxed pace I hereby propose to allow illegal reflective access > >> from code on the class path by default in JDK 9, and to disallow it in > >> a future release. > >> > >> In short, the existing "big kill switch" of the `--permit-illegal-access` > >> option [1] will become the default behavior of the JDK 9 run-time system, > >> though without as many warnings. The current behavior of JDK 9, in which > >> illegal reflective-access operations from code on the class path are not > >> permitted, will become the default in a future release. Nothing will > >> change at compile time. > >> > >> In detail, the recently-introduced `--permit-illegal-access` option will > >> be replaced by a more-general option, `--illegal-access`. This option > >> will take a single keyword parameter, as follows: > >> > >> `--illegal-access=permit` > >> > >> This will be the default mode for JDK 9. It opens every package in > >> every explicit module to code in all unnamed modules, i.e., code on > >> the class path, just as `--permit-illegal-access` does today. > >> > >> The first illegal reflective-access operation causes a warning to be > >> issued, as with `--permit-illegal-access`, but no warnings are issued > >> after that point. This single warning will describe how to enable > >> further warnings. > >> > >> `--illegal-access=warn` > >> > >> This causes a warning message to be issued for each illegal > >> reflective-access operation. This is equivalent to the current > >> `--permit-illegal-access` option. > >> > >> `--illegal-access=debug` > >> > >> This causes both a warning message and a stack trace to be shown > >> for each illegal reflective-access operation. This is equivalent > >> to combining today's `--permit-illegal-access` option with > >> `-Dsun.reflect.debugModuleAccessChecks`. > >> > >> `--illegal-access=deny` > >> > >> This disables all illegal reflective-access operations except for > >> those enabled by other command-line options, such as `--add-opens`. > >> This will become the default mode in a future release. > >> > >> Notes: > >> > >> - The proposed default mode enables the run-time system to issue a > >> warning message, possibly at some time long after startup, without > >> having been explicitly requested to do so. This may be a surprise > >> in production environments, since it's extremely unusual for the > >> run-time system to issue any warning messages at all. If the default > >> mode permits illegal reflective access, however, then it's essential > >> to make that known so that people aren't surprised when this is no > >> longer the default mode in a future release. > >> > >> - Warning messages in any mode can be avoided, as before, by the > >> judicious use of the `--add-exports` and `--add-opens` options. > >> > >> - This proposal will, if adopted, require adjustments to JEP 260, > >> "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > >> JDK will still be strongly encapsulated from the standpoint of code > >> in modules, whether those modules are automatic or explicit, but they > >> will not appear to be encapsulated at run time from the standpoint of > >> code on the class path. > >> > >> - When `deny` becomes the default mode then I expect `permit` to remain > >> supported for at least one release, so that developers can continue > >> to migrate their code. The `permit`, `warn`, and `debug` modes will, > >> over time, be removed, as will the `--illegal-access` option itself. > >> (For launch-script compatibility the unsupported modes will most > >> likely just be ignored, after issuing a warning to that effect.) > >> > >> - This change will not magically solve every JDK 9 adoption problem. > >> The concrete types of the built-in class loaders are still different, > >> `rt.jar` is still gone, the layout of a system image is still not the > >> same, and the version string still has a new format. > >> > >> Comments? > >> > >> - Mark > >> > >> > >> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > >> [2] http://openjdk.java.net/jeps/260 > >> > > > > -- > > > > PGP Key: > > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > > > Web: > > http://codefx.org > > a blog about software development > > https://www.sitepoint.com/java > > high-quality Java/JVM content > > http://do-foss.de > > Free and Open Source Software for the City of Dortmund > > > > Twitter: > > https://twitter.com/nipafx > > From forax at univ-mlv.fr Fri May 19 06:21:55 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 19 May 2017 08:21:55 +0200 (CEST) Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: <902562691.3014903.1495174915199.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stephan Herrmann" > ?: "Remi Forax" , jigsaw-dev at openjdk.java.net > Envoy?: Jeudi 18 Mai 2017 10:59:09 > Objet: Re: An alternative to "restricted keywords" + helping automatic modules > Remi, Stephan, > > I see your proposal as a minimal compromise, avoiding the worst > of difficulties, but I think we can do better. better is usually a bitter enemy > > Trade-off: > In all posts I could not find a real reason against escaping, > aside from aesthetics. I don't see this as sufficient motivation > for a less-then-perfect solution. > > > Clarity: > I'm still not completely following your explanations, partly because > of the jargon you are using. I'll leave it to Alex to decide if he > likes the idea that JLS would have to explain terms like dotted > production. Sorry for the jargon, dotted production is the same thing as a LR parser item [1], the dot mark the parsing position inside a production. i used 'dotted production' instead of parser item because usually it's clearer for my students. > > Compare this to just adding a few more rules to the grammar, > where no hand-waving is needed for an explanation. > No, I did not say that escaping is a pervasive change. > I never said that the grammar for ordinary compilation units > should be changed. > If you like we only need to extend one rule for the scope of > modular compilation units: Identifier. It can't get simpler. > I do not like ^ because - as John said, esthetics is important - it pushes the burden to the developers and not to the guys that implements the grammar. I'm Ok to makes your life and the life of people that implement the Java grammar (me included) less fun if for all other Java developers it just works, given the scale of the Java community, it seems to be a good compromise. - ^ has to be a pervasive change, i mean it can be specified as a change only for module-info but from the developers point of view, it will be weird if you introduce ^ in module-info and not introduce it in the whole grammar so it's a global solution to local problem. so in my opinion, it's not that ^ does not work, as you said, it works in Xtend, it's that ^ is a escape hatch, it's better to use it when all other solutions do not work. > > Completeness: > I understand you as saying, module names cannot start with > "transitive". Mind you, that every modifier that will be added > to the grammar for modules in the future will cause conflicts for > names that are now legal, and you won't have a means to resolve this. > > By contrast, we can use the escaping approach even to solve one > more problem that has been briefly touched on this list before: > > Automatic modules suffer from the fact that some artifact names may > have Java keywords in their name, which means that these artifacts > simply cannot be used as automatic modules, right? > Why not apply escaping also here? *Any* dot-separated sequence > of words could be used as module name, as long as module references > have a means to escape any keywords in that sequence. > > > Suitability for implementation: > As said, your proposal resolves one problem, but still IDE > functionality suffers from restricted keywords, because scanning > and parsing need more context information than normal. > - Recovery after a syntax error will regress. Error recovery will not regress in all existing java file because restricted keyword only works when parsing the module-info. And technically, there is no regression possible because the module-info was not existing before. So error recovery after a syntax error in a module-info may be less fun to handle, as i said above, i'm ok with that. > - Scanning arbitrary regions of code is not possible. Scanning an arbitrary region is not easy in general, by example, you have if you are inside or outside a string, so you have to keep some information to be able to scan a region, why not trying to keep the parser state when necessary. As John said, it seems to be a nice problem for grad students and at worst, you can use the existing code, it will display a restricted keyword in bold in the middle of a package name, that's all. > Remember: > In an IDE code with syntax errors is the norm, not an exception, > as the IDE provides functionality to work on incomplete code. > > > Stephan R?mi [1] https://en.wikipedia.org/wiki/Canonical_LR_parser > > > On 18.05.2017 00:34, Remi Forax wrote: >> I want to answer this before we start the meetings because i really think that >> restricted keyword as i propose solve the issues Stephan raised. >> >> >> ----- Mail original ----- >>> De: "Stephan Herrmann" >>> ?: jigsaw-dev at openjdk.java.net >>> Envoy?: Mardi 16 Mai 2017 11:49:45 >>> Objet: Re: An alternative to "restricted keywords" >> >>> Thanks, Remi, for taking this to the EG list. >>> >>> Some collected responses: >>> >>> >>> Remi: "from the user point of view, '^' looks like a hack" >>> >>> This is, of course, a subjective statement. I don't share this view >>> and in years of experience with Xtext-languages (where this concept >>> is used by default) I never heard any user complain about this. >>> >>> More importantly, I hold that such aesthetic considerations are of >>> much lesser significance than the question, whether we can explain >>> - unambiguously explain - the concept in a few simple sentences. >>> Explaining must be possible at two levels: in a rigorous specification >>> and in simple words for users of the language. >> >> I'm not against ^, or ` as it has already asked to escape an identifier, but as >> you said it's a pervasive change that applies on the whole grammar while i >> think that with restricted keyword (that really should be called local >> keywords) the changes only impact the grammar that specifies a module-info.java >> >>> >>> Remi: "a keyword which is activated if you are at a position in the >>> grammar where it can be recognized". >>> >>> I don't think 'being at a position in the grammar' is a good way of >>> explaining. Parsing doesn't generally have one position in a grammar, >>> multiple productions can be active in the same parser state. >>> Also speaking of a "loop" for modifiers seems to complicate matters >>> more than necessary. >>> >>> Under these considerations I still see '^' as the clearest of all >>> solutions. Clear as a specification, simple to explain to users. >> >> Eclipse uses a LR parser, for a LR parser, position == dotted production as i >> have written earlier, so no problem because it corresponds to only one parser >> state. Note that even if one do not use an LR or a LL parser, most hand >> written parser i've seen, javac is one of them, also refers to dotted >> production in the comments of the corresponding methods. >> >>> >>> >>> >>> Peter spoke about module names vs. package names. >>> >>> I think we agree, that module names cannot use "module words", >>> whereas package names should be expected to contain them. >> >> yes, that the main issue, package names may contains unqualified name like >> 'transitive, ''with' or 'to'. >> but i think people will also want to use existing package or more exactly prefix >> of existing package as module name, so we should also support having restricted >> keyword name as part of a module name. >> >> The grammar is: >> >> open? module module_name { >> requires (transitive | static)* module_name; >> exports package_name; >> exports package_name to module_name1, module_name2; >> opens package_name; >> opens package_name to module_name1, module_name2; >> uses xxx; >> provides xxx with xxx, yyy; >> } >> >> If we just consider package name, only 'opens' and 'exports' are followed by a >> package name and a package name can only been followed by ';' or 'to', so once >> 'opens' is parsed, you know that you can have only an identifier so if it's not >> an identifier by one of the restricted keywords, it should be considered as an >> identifier. >> >> As i said earlier, the scanner can see the restricted keyword as keyword and >> before feeding the token to the parser, you can check the parser state to see >> if the keyword as to be lowered to an identifier or not. >> >> For module name, there is the supplementary problem of transitive, because if a >> module starts with transitive, you can have a conflict. As i said earlier, >> instead of using the next token to know if transitive is the keyword or part of >> the module name, i think we should consider it as a keyword, as the JLS said a >> restricted keyword is activated when it can appear, so "requires transitive" is >> not a valid directive. >> >>> >>> Remi: "you should use reverse DNS naming for package so no problem :)" >>> >>> "to" is a "module word" and a TLD. >>> I think we should be very careful in judging that a existing conflict >>> is not a real problem. Better to clearly and rigorously avoid the >>> conflict in the first place. >> >> to as the first part of a package/module and to as in exports ... to can not be >> present on the same dotted production, because exports as to be followed by a >> package_name so 'to' here means the start of a package name and then because a >> package name can not ends with '.' you always know if you are inside the >> production recognizing the package_name or outside matching the to of the >> directive exports. >> >>> >>> >>> >>> Some additional notes from my side: >>> >>> In the escape-approach, it may be prudent to technically allow >>> escaping even words that are identifiers in Java 9, but could become >>> keywords in a future version. This ensures that modules which need >>> more escaping in Java 9+X can still be parsed in Java 9. >> >> yes, that's why i think that escaping is not the right mechanism here, because >> we want to solve a very local problem so we do not need a global grammar-wise >> way to solve our problem. >> >>> >>> >>> Current focus was on names of modules, packages and types. >>> A complete solution must also give an answer for annotations on modules. >>> Some possible solutions: >>> a. Assume that annotations for modules are designed with modules in mind >>> and thus have to avoid any module words in their names. >>> b. Support escaping also in annotations >>> c. Refine the scope where "module words" are keywords, let it start only >>> when the word "module" or the group "open module" has been consumed. >>> This would make the words "module" and "open" special, as being >>> switch words, where we switch from one language to another. >>> (For this I previously coined the term "scoped keywords" [1]) >> >> For annotation, again, because annotation name are qualified, you now when you >> see 'module' if you are in the middle of the annotation name or if you are >> outside. >> >>> >>> >>> I think we all agree that the conflicts we are solving here are rare >>> corner cases. Most names do not contain module words. Still, from a >>> conceptual and technical p.o.v. the solution must be bullet proof. >>> But there's no need to be afraid of module declarations being spammed >>> with dozens of '^' characters. Realistically, this will not happen. >>> >> >> I agree, and i strongly believe that scoped keyword, local keywords or >> restricted keywords, i.e. whatever the name, keywords that are keywords or >> identifiers depending on the parser state are the general mechanism that solve >> our problem. >> >>> Stephan >>> >>> [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 >>> >> >> R?mi From rafael.wth at gmail.com Fri May 19 07:08:02 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 09:08:02 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: Hi Alan, I just retested with the most recent Java 9 snapshot and this is what I get: rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar -m com.greetings/com.greetings.Main Error occurred during initialization of VM Could not find agent library instrument on the library path, with error: libinstrument.so: cannot open shared object file: No such file or directory rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules java.instrument -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar -m com.greetings/com.greetings.Main Error occurred during initialization of VM Could not find agent library instrument on the library path, with error: libinstrument.so: cannot open shared object file: No such file or directory rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules java.instrument -m com.greetings/com.greetings.MainError occurred during initialization of boot layer java.lang.module.FindException: Module java.instrument not found Do I understand you correctly that you expect the last outcome also as a result for the first command? I argue that this outcome would still be rather suprising for most Java users. Many monitoring tools use Java agents to hook into Java processes and those tools have become omnipresent in most production environments. Also, it is very unlikely that a main application module would require the java.instrument module as it is only useful to Java agents. As a result, Java agents and the tooling that relies on them would stop working for the majority of jlink modules. Furthermore, the agent attachment also fails if a Javaagent does not require the Instrumentation interface what makes this problem even less intuitive. (The reason being the lack of libinstrument.) Consider the follwoing runtime environment which I have seen at customers dozens of times: rafael at rafc:~/jdk9-test/greetingsapp$ export _JAVA_OPTIONS=-javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java -m com.greetings/com.greetings.Main Picked up _JAVA_OPTIONS: -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar Error occurred during initialization of VM Could not find agent library instrument on the library path, with error: libinstrument.so: cannot open shared object file: No such file or directory On the above machine, a jlink image is incapable of starting despite the _JAVA_OPTIONS only specifying standard features that are guaranteed by the JVM independently of a specific VM vendor. To resolve this, I think that the jlink tools should always resolve the java.instrument module. Right now, it is most likely creating a JVM image that is not compatible to the majority of existing tooling, including the tooling provided by Oracle. Other than the experimental options and the management extension, the Java agent mechanism is part of the officially documented options which is supported by any JVM implementation (e.g. it is listed by java --help). That this feature is practically unavailable unless the java.instrument module is resolved (which might just happen accidentally when a certain module requires java.instrument transitively) could backfire when discovered late, for example when it is impossible to debug a production process when it yields an error after a few weeks that would require introspection of the process which is usually done using a Java agent. In the worst case, an application could be distributed by a vendor who is not aware of tooling being used to monitor Java processes where building the image is not governed by those relying on java.instrument. I further argue that java.instrument should be part of java.base what might be the easiest solution to this problem. The instrumentation API is a core feature to any JVM and any Java agent should be able to start on any JVM distribution. If a Java agent requires APIs that is not provided by a JVM image due to missing modules beyond java.instrument, the Javaagent can in such cases detect such lacking functionality at runtime and choose to fail with a proper error message or to offer some reduced functionality. The java.instrument types however are part of the startup signature which is why they should always be included. I really hope you consider this. Java agents often seem like a feature used by only a few people but I have rarely seen a production environment where they are not used in some form. In the end, this would also affect Oracle customers and Oracle products. Best regards, Rafael 2017-05-18 17:19 GMT+02:00 Alan Bateman : > On 18/05/2017 15:20, Rafael Winterhalter wrote: > > Hei, >> >> I found that it is impossible to dynamically attach to a JVM that does not >> include the java.instrument module when built with jlink. This is a result >> of the instrumentation API and its infrastructure missing from the image. >> >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java >> -javaagent:/home/rafael/.m2/repository/sample-agent/sample- >> agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar >> -m com.greetings/com.greetings.Main >> Error occurred during initialization of VM >> >> Could not find agent library instrument on the library path, with error: >> libinstrument.so: cannot open shared object file: No such file or >> directory >> > Ugh, when -javaagent is specified then it is supposed to work as if > `--add-modules java.instrument` is also on the command line. This ensures > that the java.instrument module is resolved. The intention was also to get > a useful error if the run-time image doesn't contain java.instrument. > > Another example is `-Dcom.sun.management.*` which should work as if > `--add-modules jdk.management.agent` is specified on the command line. Also > `-XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI` which is supposed to > add `--add-modules jdk.internal.vm.ci`. I just checked both of these and > the error is helpful as I expected. > > As regards the attach mechanism then it will work when it is compiled in > and enabled in the target VM. When I say "compiled in" then I mean the > support is in libjvm. When using jlink when you can select the minimal VM > to get a much smaller VM that doesn't have the serviceability features. > That won't be typical of course so assume it is libjvm then you should be > able to attach and troubleshooting tools such as `jcmd` will work. If you > attempt to load a java agent then the target VM will attempt to resolve and > load the java.instrument module and so will fail if the run-time image > doesn't have the module. > > You are right that java.instrument is small but it's always been a goal to > be able to create run-time images that only contain java.base. > > I didn't understand your comment about an "unused warnings" in the IDE. Do > you mean `requires java.instrument` rather than specifying java.instrument > to the `jlink --add-modules` option? > > -Alan > From Alan.Bateman at oracle.com Fri May 19 08:22:04 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 09:22:04 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: On 19/05/2017 08:08, Rafael Winterhalter wrote: > Hi Alan, > > I just retested with the most recent Java 9 snapshot and this is what > I get: > > rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java > -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar > -m com.greetings/com.greetings.Main > Error occurred during initialization of VM > Could not find agent library instrument on the library path, with > error: libinstrument.so: cannot open shared object file: No such file > or directory > rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules > java.instrument > -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar > -m com.greetings/com.greetings.Main > Error occurred during initialization of VM > Could not find agent library instrument on the library path, with > error: libinstrument.so: cannot open shared object file: No such file > or directory > rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules > java.instrument -m com.greetings/com.greetings.MainError occurred > during initialization of boot layer > java.lang.module.FindException: Module java.instrument not found > > Do I understand you correctly that you expect the last outcome also as > a result for the first command? If the run-time image does not contain java.instrument then the error for each of these cases should be like the last one ("Module java.instrument not found"). We have it right for -Dcom.sun.management.* and -XX:+EnableJVMCI but not -javaagent. The reason for the strange error with -javaagent is because java agents are implemented as a JVM TI agent that is loaded early in VM startup, long before the root modules are resolved. I will create a bug for this. > > I argue that this outcome would still be rather suprising for most > Java users. Many monitoring tools use Java agents to hook into Java > processes and those tools have become omnipresent in most production > environments. Also, it is very unlikely that a main application module > would require the java.instrument module as it is only useful to Java > agents. As a result, Java agents and the tooling that relies on them > would stop working for the majority of jlink modules. Furthermore, the > agent attachment also fails if a Javaagent does not require the > Instrumentation interface what makes this problem even less intuitive. > (The reason being the lack of libinstrument.) The jlink user is a power user and so decides the modules and features to include in the run-time image. If the jlink user doesn't select the VM type when creating the run-time image then the attach mechanism will work (because it is compiled into libjvm.so) and the troubleshooting tools should work fine. However some features that are exposed via the attach mechanism may not be present, specifically VirtualMachine.startManagementAgent and VirtualMachine.startLocalManagementAgent will fail if module jdk.management.agent is not included, and VirtualMachine.loadAgent will fail if module java.instrument is not included. If the jlink user is targeting an embedded system or wants to create a minimal image to put in a docker container then maybe they select the minimal VM, in which case the serviceability features (including the VM side of the attach mechanism) are not included. It gets harder again for troubleshooting when additional jlink power options are used to strip debug attributes. That's the trade-off. As I said, the goal has always been to allow someone create a run-time image that only contains java.base. I'm not sure that subsuming java.instrument into java.base is right. Introducing options to ask jlink to exclude modules creates the potential for conflict on the command line. One thing that jlink could do is emit a warning that the resulting run-time image doesn't have the management and instrumentation features, might that be the right balance. > > On the above machine, a jlink image is incapable of starting despite > the _JAVA_OPTIONS only specifying standard features that are > guaranteed by the JVM independently of a specific VM vendor. There isn't any guarantee, a point that was clarified in the java.instrument spec in Java SE 8 (JDK-8006565). The motivation for the clarification at the time was embedded deployments and the Compact Profiles defined for Java SE 8 (compact1 and compact2 didn't include the java.lang.instrument package). -Alan From forax at univ-mlv.fr Fri May 19 08:40:18 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 19 May 2017 10:40:18 +0200 (CEST) Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: <972121387.3144678.1495183218703.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alan Bateman" > ?: "Rafael Winterhalter" > Cc: "jigsaw-dev" > Envoy?: Vendredi 19 Mai 2017 10:22:04 > Objet: Re: Attaching to a JVM image that does not include java.instrument > On 19/05/2017 08:08, Rafael Winterhalter wrote: > >> Hi Alan, >> >> I just retested with the most recent Java 9 snapshot and this is what >> I get: >> >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java >> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar >> -m com.greetings/com.greetings.Main >> Error occurred during initialization of VM >> Could not find agent library instrument on the library path, with >> error: libinstrument.so: cannot open shared object file: No such file >> or directory >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules >> java.instrument >> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar >> -m com.greetings/com.greetings.Main >> Error occurred during initialization of VM >> Could not find agent library instrument on the library path, with >> error: libinstrument.so: cannot open shared object file: No such file >> or directory >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules >> java.instrument -m com.greetings/com.greetings.MainError occurred >> during initialization of boot layer >> java.lang.module.FindException: Module java.instrument not found >> >> Do I understand you correctly that you expect the last outcome also as >> a result for the first command? > If the run-time image does not contain java.instrument then the error > for each of these cases should be like the last one ("Module > java.instrument not found"). We have it right for -Dcom.sun.management.* > and -XX:+EnableJVMCI but not -javaagent. The reason for the strange > error with -javaagent is because java agents are implemented as a JVM TI > agent that is loaded early in VM startup, long before the root modules > are resolved. I will create a bug for this. >> >> I argue that this outcome would still be rather suprising for most >> Java users. Many monitoring tools use Java agents to hook into Java >> processes and those tools have become omnipresent in most production >> environments. Also, it is very unlikely that a main application module >> would require the java.instrument module as it is only useful to Java >> agents. As a result, Java agents and the tooling that relies on them >> would stop working for the majority of jlink modules. Furthermore, the >> agent attachment also fails if a Javaagent does not require the >> Instrumentation interface what makes this problem even less intuitive. >> (The reason being the lack of libinstrument.) > The jlink user is a power user and so decides the modules and features > to include in the run-time image. > > If the jlink user doesn't select the VM type when creating the run-time > image then the attach mechanism will work (because it is compiled into > libjvm.so) and the troubleshooting tools should work fine. However some > features that are exposed via the attach mechanism may not be present, > specifically VirtualMachine.startManagementAgent and > VirtualMachine.startLocalManagementAgent will fail if module > jdk.management.agent is not included, and VirtualMachine.loadAgent will > fail if module java.instrument is not included. > > If the jlink user is targeting an embedded system or wants to create a > minimal image to put in a docker container then maybe they select the > minimal VM, in which case the serviceability features (including the VM > side of the attach mechanism) are not included. It gets harder again for > troubleshooting when additional jlink power options are used to strip > debug attributes. That's the trade-off. > > As I said, the goal has always been to allow someone create a run-time > image that only contains java.base. I'm not sure that subsuming > java.instrument into java.base is right. Introducing options to ask > jlink to exclude modules creates the potential for conflict on the > command line. One thing that jlink could do is emit a warning that the > resulting run-time image doesn't have the management and instrumentation > features, might that be the right balance. yes, that seems the right way to solve this issue. > >> >> On the above machine, a jlink image is incapable of starting despite >> the _JAVA_OPTIONS only specifying standard features that are >> guaranteed by the JVM independently of a specific VM vendor. > There isn't any guarantee, a point that was clarified in the > java.instrument spec in Java SE 8 (JDK-8006565). The motivation for the > clarification at the time was embedded deployments and the Compact > Profiles defined for Java SE 8 (compact1 and compact2 didn't include the > java.lang.instrument package). > > -Alan R?mi From scolebourne at joda.org Fri May 19 08:58:01 2017 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 19 May 2017 09:58:01 +0100 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: I don't support the ^ element or escaping like that either. However, would adding a "module" keyword help? module com.foo.lib { requires module com.foo.bar; } thus: module com.foo.lib { requires static module blah; requires transitive module transitive; } ie. the module name is always prefixed by "module" in a "requires" statement. But does this help? Stephen On 18 May 2017 at 09:59, Stephan Herrmann wrote: > Remi, > > I see your proposal as a minimal compromise, avoiding the worst > of difficulties, but I think we can do better. > > Trade-off: > In all posts I could not find a real reason against escaping, > aside from aesthetics. I don't see this as sufficient motivation > for a less-then-perfect solution. > > > Clarity: > I'm still not completely following your explanations, partly because > of the jargon you are using. I'll leave it to Alex to decide if he > likes the idea that JLS would have to explain terms like dotted > production. > > Compare this to just adding a few more rules to the grammar, > where no hand-waving is needed for an explanation. > No, I did not say that escaping is a pervasive change. > I never said that the grammar for ordinary compilation units > should be changed. > If you like we only need to extend one rule for the scope of > modular compilation units: Identifier. It can't get simpler. > > > Completeness: > I understand you as saying, module names cannot start with > "transitive". Mind you, that every modifier that will be added > to the grammar for modules in the future will cause conflicts for > names that are now legal, and you won't have a means to resolve this. > > By contrast, we can use the escaping approach even to solve one > more problem that has been briefly touched on this list before: > > Automatic modules suffer from the fact that some artifact names may > have Java keywords in their name, which means that these artifacts > simply cannot be used as automatic modules, right? > Why not apply escaping also here? *Any* dot-separated sequence > of words could be used as module name, as long as module references > have a means to escape any keywords in that sequence. > > > Suitability for implementation: > As said, your proposal resolves one problem, but still IDE > functionality suffers from restricted keywords, because scanning > and parsing need more context information than normal. > - Recovery after a syntax error will regress. > - Scanning arbitrary regions of code is not possible. > Remember: > In an IDE code with syntax errors is the norm, not an exception, > as the IDE provides functionality to work on incomplete code. > > > Stephan > > > On 18.05.2017 00:34, Remi Forax wrote: >> >> I want to answer this before we start the meetings because i really think >> that restricted keyword as i propose solve the issues Stephan raised. >> >> >> ----- Mail original ----- >>> >>> De: "Stephan Herrmann" >>> ?: jigsaw-dev at openjdk.java.net >>> Envoy?: Mardi 16 Mai 2017 11:49:45 >>> Objet: Re: An alternative to "restricted keywords" >> >> >>> Thanks, Remi, for taking this to the EG list. >>> >>> Some collected responses: >>> >>> >>> Remi: "from the user point of view, '^' looks like a hack" >>> >>> This is, of course, a subjective statement. I don't share this view >>> and in years of experience with Xtext-languages (where this concept >>> is used by default) I never heard any user complain about this. >>> >>> More importantly, I hold that such aesthetic considerations are of >>> much lesser significance than the question, whether we can explain >>> - unambiguously explain - the concept in a few simple sentences. >>> Explaining must be possible at two levels: in a rigorous specification >>> and in simple words for users of the language. >> >> >> I'm not against ^, or ` as it has already asked to escape an identifier, >> but as you said it's a pervasive change that applies on the whole grammar >> while i think that with restricted keyword (that really should be called >> local keywords) the changes only impact the grammar that specifies a >> module-info.java >> >>> >>> Remi: "a keyword which is activated if you are at a position in the >>> grammar where it can be recognized". >>> >>> I don't think 'being at a position in the grammar' is a good way of >>> explaining. Parsing doesn't generally have one position in a grammar, >>> multiple productions can be active in the same parser state. >>> Also speaking of a "loop" for modifiers seems to complicate matters >>> more than necessary. >>> >>> Under these considerations I still see '^' as the clearest of all >>> solutions. Clear as a specification, simple to explain to users. >> >> >> Eclipse uses a LR parser, for a LR parser, position == dotted production >> as i have written earlier, so no problem because it corresponds to only one >> parser state. Note that even if one do not use an LR or a LL parser, most >> hand written parser i've seen, javac is one of them, also refers to dotted >> production in the comments of the corresponding methods. >> >>> >>> >>> >>> Peter spoke about module names vs. package names. >>> >>> I think we agree, that module names cannot use "module words", >>> whereas package names should be expected to contain them. >> >> >> yes, that the main issue, package names may contains unqualified name like >> 'transitive, ''with' or 'to'. >> but i think people will also want to use existing package or more exactly >> prefix of existing package as module name, so we should also support having >> restricted keyword name as part of a module name. >> >> The grammar is: >> >> open? module module_name { >> requires (transitive | static)* module_name; >> exports package_name; >> exports package_name to module_name1, module_name2; >> opens package_name; >> opens package_name to module_name1, module_name2; >> uses xxx; >> provides xxx with xxx, yyy; >> } >> >> If we just consider package name, only 'opens' and 'exports' are followed >> by a package name and a package name can only been followed by ';' or 'to', >> so once 'opens' is parsed, you know that you can have only an identifier so >> if it's not an identifier by one of the restricted keywords, it should be >> considered as an identifier. >> >> As i said earlier, the scanner can see the restricted keyword as keyword >> and before feeding the token to the parser, you can check the parser state >> to see if the keyword as to be lowered to an identifier or not. >> >> For module name, there is the supplementary problem of transitive, because >> if a module starts with transitive, you can have a conflict. As i said >> earlier, instead of using the next token to know if transitive is the >> keyword or part of the module name, i think we should consider it as a >> keyword, as the JLS said a restricted keyword is activated when it can >> appear, so "requires transitive" is not a valid directive. >> >>> >>> Remi: "you should use reverse DNS naming for package so no problem :)" >>> >>> "to" is a "module word" and a TLD. >>> I think we should be very careful in judging that a existing conflict >>> is not a real problem. Better to clearly and rigorously avoid the >>> conflict in the first place. >> >> >> to as the first part of a package/module and to as in exports ... to can >> not be present on the same dotted production, because exports as to be >> followed by a package_name so 'to' here means the start of a package name >> and then because a package name can not ends with '.' you always know if you >> are inside the production recognizing the package_name or outside matching >> the to of the directive exports. >> >>> >>> >>> >>> Some additional notes from my side: >>> >>> In the escape-approach, it may be prudent to technically allow >>> escaping even words that are identifiers in Java 9, but could become >>> keywords in a future version. This ensures that modules which need >>> more escaping in Java 9+X can still be parsed in Java 9. >> >> >> yes, that's why i think that escaping is not the right mechanism here, >> because we want to solve a very local problem so we do not need a global >> grammar-wise way to solve our problem. >> >>> >>> >>> Current focus was on names of modules, packages and types. >>> A complete solution must also give an answer for annotations on modules. >>> Some possible solutions: >>> a. Assume that annotations for modules are designed with modules in mind >>> and thus have to avoid any module words in their names. >>> b. Support escaping also in annotations >>> c. Refine the scope where "module words" are keywords, let it start only >>> when the word "module" or the group "open module" has been consumed. >>> This would make the words "module" and "open" special, as being >>> switch words, where we switch from one language to another. >>> (For this I previously coined the term "scoped keywords" [1]) >> >> >> For annotation, again, because annotation name are qualified, you now when >> you see 'module' if you are in the middle of the annotation name or if you >> are outside. >> >>> >>> >>> I think we all agree that the conflicts we are solving here are rare >>> corner cases. Most names do not contain module words. Still, from a >>> conceptual and technical p.o.v. the solution must be bullet proof. >>> But there's no need to be afraid of module declarations being spammed >>> with dozens of '^' characters. Realistically, this will not happen. >>> >> >> I agree, and i strongly believe that scoped keyword, local keywords or >> restricted keywords, i.e. whatever the name, keywords that are keywords or >> identifiers depending on the parser state are the general mechanism that >> solve our problem. >> >>> Stephan >>> >>> [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 >>> >> >> R?mi >> > From rafael.wth at gmail.com Fri May 19 09:20:27 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 11:20:27 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: While the jlink feature might only be used by power users, there is no guarantee that the users of these images are power users at the same degree or is even understand the implications. For example, some team might use a JVM-based server from a third-party vendor which is distributed as a jlink image and deploy it in a production environment. The operations team is monitoring deployed processes by different mechanisms. To monitor Java applications, production teams typically use agent-based tooling that reports certain metrics and a heart beat. If these metrics are not provided by the agent, the environment considers the application to be non-responsive, kills the process and restarts it. The development team is therefore asked to enable the Java agent attachment for the server application which they do however not control but the third-party vendor which would now need to change their product in order to satisfy the monitoring requirements of one of their customers what they did of course not anticipate. It is the exact purpose of a Java agent to allow the enhancement of a Java application that is independent of a particular Java process. Therefore, it should not be a decission bound to the creation or bundeling of the Java application if such user-site monitoring is required. Instead, it should always be possible to attach a Java agent for a standalone Java process. Also, I cannot find any comment about the Java agent functionality to be optional despite the mentioned change set: https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html - Neither would anybody using the functionality expect it to be optional. I work with agents everyday and this is the first time I hear it. Finally, the major gain of not always bundeling the java.instrument module with a jlink image is the reduced size. On my machine, excluding java.instrument yields a difference of 131 kB. This is close to irrelevant even when working with Docker images. Is this size reduction really worth jeopardizing a feature that many people rely on? In the end, the base size of an image is about 45 MB, I do not see how this reduction matters given the sacrifice. For all of these reasons, I see no good reason why the java.instrument module should not be merged into the java.base module. Even in embedded environments, if for example Android wanted to adopt the JPMS, the presense of 5 more interfaces would not matter much even if there is no way of attaching an agent and the libinstrument native code could still be dropped. At the same time, agents would continue to work reliably for any JVM process as most people expect it today and this important corner stone of easy access to monitoring all JVM processes would still be guaranteed. 2017-05-19 10:22 GMT+02:00 Alan Bateman : > On 19/05/2017 08:08, Rafael Winterhalter wrote: > > Hi Alan, >> >> I just retested with the most recent Java 9 snapshot and this is what I >> get: >> >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java >> -javaagent:/home/rafael/.m2/repository/sample-agent/sample- >> agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar -m com.greetings/ >> com.greetings.Main >> Error occurred during initialization of VM >> Could not find agent library instrument on the library path, with error: >> libinstrument.so: cannot open shared object file: No such file or directory >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules >> java.instrument -javaagent:/home/rafael/.m2/re >> pository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar >> -m com.greetings/com.greetings.Main >> Error occurred during initialization of VM >> Could not find agent library instrument on the library path, with error: >> libinstrument.so: cannot open shared object file: No such file or directory >> rafael at rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules >> java.instrument -m com.greetings/com.greetings.MainError occurred during >> initialization of boot layer >> java.lang.module.FindException: Module java.instrument not found >> >> Do I understand you correctly that you expect the last outcome also as a >> result for the first command? >> > If the run-time image does not contain java.instrument then the error for > each of these cases should be like the last one ("Module java.instrument > not found"). We have it right for -Dcom.sun.management.* and > -XX:+EnableJVMCI but not -javaagent. The reason for the strange error with > -javaagent is because java agents are implemented as a JVM TI agent that is > loaded early in VM startup, long before the root modules are resolved. I > will create a bug for this. > >> >> I argue that this outcome would still be rather suprising for most Java >> users. Many monitoring tools use Java agents to hook into Java processes >> and those tools have become omnipresent in most production environments. >> Also, it is very unlikely that a main application module would require the >> java.instrument module as it is only useful to Java agents. As a result, >> Java agents and the tooling that relies on them would stop working for the >> majority of jlink modules. Furthermore, the agent attachment also fails if >> a Javaagent does not require the Instrumentation interface what makes this >> problem even less intuitive. (The reason being the lack of libinstrument.) >> > The jlink user is a power user and so decides the modules and features to > include in the run-time image. > > If the jlink user doesn't select the VM type when creating the run-time > image then the attach mechanism will work (because it is compiled into > libjvm.so) and the troubleshooting tools should work fine. However some > features that are exposed via the attach mechanism may not be present, > specifically VirtualMachine.startManagementAgent and > VirtualMachine.startLocalManagementAgent will fail if module > jdk.management.agent is not included, and VirtualMachine.loadAgent will > fail if module java.instrument is not included. > > If the jlink user is targeting an embedded system or wants to create a > minimal image to put in a docker container then maybe they select the > minimal VM, in which case the serviceability features (including the VM > side of the attach mechanism) are not included. It gets harder again for > troubleshooting when additional jlink power options are used to strip debug > attributes. That's the trade-off. > > As I said, the goal has always been to allow someone create a run-time > image that only contains java.base. I'm not sure that subsuming > java.instrument into java.base is right. Introducing options to ask jlink > to exclude modules creates the potential for conflict on the command line. > One thing that jlink could do is emit a warning that the resulting run-time > image doesn't have the management and instrumentation features, might that > be the right balance. > > >> On the above machine, a jlink image is incapable of starting despite the >> _JAVA_OPTIONS only specifying standard features that are guaranteed by the >> JVM independently of a specific VM vendor. >> > There isn't any guarantee, a point that was clarified in the > java.instrument spec in Java SE 8 (JDK-8006565). The motivation for the > clarification at the time was embedded deployments and the Compact Profiles > defined for Java SE 8 (compact1 and compact2 didn't include the > java.lang.instrument package). > > -Alan > > From volker.simonis at gmail.com Fri May 19 09:48:23 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 19 May 2017 11:48:23 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: Hi Mark, thanks for this proposal. I think it is pragmatic and well balanced. While I personally agree with the advocates of a clean solution I also understand the the needs of big (and slow) companies (including the one I'm working for:) After all, the new default isn't that bad in my opinion because it only affects applications/libraries on the class path which is legacy anyway. If you convert your application/libraries to modules it will still benefit from the full JPMS features and restrictions. So altogether I think it is a good compromise. I have some practical comments/suggestions though: >From my understanding, at run-time, "open" implicates "exports" (i.e. if a module M1 opens a package P for some other module M2 it also, implicitly exports P to M2). The "big kill switch" in both, its old and in the newly proposed form, usually only refers to "enabling reflective access" but doesn't explicitly mentions that it will also, implicitly export the respective packages. Also, the usage of the "kill switch" only produces warnings for reflective accesses which are enabled by the option itself (and not at the same time, explicitly allowed by --add-opens directives). But it doesn't warn about the simple, non-reflective accesses to packages which are implicitly exported by the kill switch as well. I think that the "big kill switch" should also print warnings for these simple, non-reflective accesses to packages which are implicitly exported by the kill switch. Otherwise, this could lead to a situation where users claim to be "JPMS-safe" because they did run with "--illegal-access=warn" and didn't saw any warnings but then, after the default will change to "--illegal-access=deny", their applications will break because of simple accesses to non-exported packages. I'm aware of the initial discussion about "--permit-illegal-access" and that you decided not to issue warnings for the precise `--add-opens` and `--add-exports` options [1]. But now, with the new version of the "--illegal-access" option and the possibility to enable/disable warnings, I think printing warnings for simple (i.e. non-reflective) illegal accesses which are implicitly allowed "--illegal-access" by would be useful. Regards, Volker [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011810.html On Thu, May 18, 2017 at 4:48 PM, wrote: > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > [2] http://openjdk.java.net/jeps/260 From peter.levart at gmail.com Fri May 19 10:11:26 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 12:11:26 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> Message-ID: <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> On 05/19/2017 01:17 AM, Nicolai Parlog wrote: > With illegal access being permitted by default much fewer developers > will be aware of the problem and much less pressure will be put on > library and framework maintainers as well as on project management to > invest into paying back this particular form of technical debt. So we > get much less momentum to make the necessary changes in exchange for... > not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the > Java community! +1 I think that --illegal-access=permit-silently or equivalent is a reasonable request (in addition to other modes), but please make the --illegal-access=deny the default! There was a lot of work put into moving various internal classes into jdk.internal.* packages (except for some widely used ones), so enabling illegal access from classpath by default allows careless developers to start using those new internal APIs (such as jdk.internal.misc.Unsafe) because they can!!! And we are back to square one when JDK 9 -> JDK 10 transition is to be made. Not only will we be forced to search for public alternatives to existing private (JDK 8-) APIs, but in addition to all jdk.internal.* APIs that were newly introduced in JDK 9 too... Regards, Peter From peter.levart at gmail.com Fri May 19 10:24:07 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 12:24:07 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> Hi Volker, On 05/19/2017 11:48 AM, Volker Simonis wrote: > From my understanding, at run-time, "open" implicates "exports" (i.e. > if a module M1 opens a package P for some other module M2 it also, > implicitly exports P to M2). The "big kill switch" in both, its old > and in the newly proposed form, usually only refers to "enabling > reflective access" but doesn't explicitly mentions that it will also, > implicitly export the respective packages. Also, the usage of the > "kill switch" only produces warnings for reflective accesses which are > enabled by the option itself (and not at the same time, explicitly > allowed by --add-opens directives). But it doesn't warn about the > simple, non-reflective accesses to packages which are implicitly > exported by the kill switch as well. No, you got it wrong. "opens" is just a backdoor for reflection (and MethodHandles.Lookup). It does not implicitly also mean "exports". Neither in compile-time nor in run-time. Otherwise everything that is declared "public" would be accessible to reflection without the need for .setAccessible(true) - therefore even to code running with SecurityManager enabled! Regards, Peter From Alan.Bateman at oracle.com Fri May 19 10:24:43 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 11:24:43 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: On 19/05/2017 10:20, Rafael Winterhalter wrote: > : > > It is the exact purpose of a Java agent to allow the enhancement of a > Java application that is independent of a particular Java process. > Therefore, it should not be a decission bound to the creation or > bundeling of the Java application if such user-site monitoring is > required. Instead, it should always be possible to attach a Java agent > for a standalone Java process. Also, I cannot find any comment about > the Java agent functionality to be optional despite the mentioned > change set: > https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html > - Neither would anybody using the functionality expect it to be > optional. I work with agents everyday and this is the first time I > hear it. > I don't have time to debate this topic just now but just on optionally, the tool interfaces have always been optional. It might need digging to find all the history but as I recall, in 2004 the then in-progress JSR-163 spec was clarified to make the optionality clear. This is why the JVM TI spec has the statement "JVM TI may not be available in all implementations of the JavaTM virtual machine". The wording in the java.lang.instrument package description wasn't as clear, something that came to light when shaking out the details of Compact Profiles for Java SE 8. This lead to a small clarification [1] that I mentioned. -Alan. [1] http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/4a4b97f7f83b From stephan.herrmann at berlin.de Fri May 19 10:37:07 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 19 May 2017 12:37:07 +0200 (CEST) Subject: An alternative to "restricted keywords" + helping automatic modules Message-ID: <1495190227.645288-11404@martha.daybyday.de> A quick question to keep the ball rolling: Do we agree on the following assessment of the status quo? The definition of "restricted keywords" implies (without explicitly saying so), that classification of a word as keyword vs. identifier can only be made *after* parsing has accepted the enclosing ModuleDeclaration. (With some tweaks, this can be narrowed down to "after the enclosing ModuleDirective has been accepted") This definition is not acceptable. comments? Stephan ----- urspr?ngliche Nachricht --------- Subject: Re: An alternative to "restricted keywords" + helping automatic modules Date: Fr 19 Mai 2017 07:27:31 CEST From: John Rose To: Stephan Herrmann On May 18, 2017, at 1:59 AM, Stephan Herrmann wrote: In all posts I could not find a real reason against escaping, aside from aesthetics. I don't see this as sufficient motivation for a less-then-perfect solution. So, by disregarding esthetics... Clarity: I'm still not completely following your explanations, partly because of the jargon you are using. I'll leave it to Alex to decide if he likes the idea that JLS would have to explain terms like dotted production. Compare this to just adding a few more rules to the grammar, where no hand-waving is needed for an explanation. No, I did not say that escaping is a pervasive change. I never said that the grammar for ordinary compilation units should be changed. If you like we only need to extend one rule for the scope of modular compilation units: Identifier. It can't get simpler. Completeness: I understand you as saying, module names cannot start with "transitive". Mind you, that every modifier that will be added to the grammar for modules in the future will cause conflicts for names that are now legal, and you won't have a means to resolve this. By contrast, we can use the escaping approach even to solve one more problem that has been briefly touched on this list before: Automatic modules suffer from the fact that some artifact names may have Java keywords in their name, which means that these artifacts simply cannot be used as automatic modules, right? Why not apply escaping also here? *Any* dot-separated sequence of words could be used as module name, as long as module references have a means to escape any keywords in that sequence. Suitability for implementation: As said, your proposal resolves one problem, but still IDE functionality suffers from restricted keywords, because scanning and parsing need more context information than normal. ?we obtain the freedom for IDEs to disregard abnormalamounts of context, saving uncounted machine cycles, - Recovery after a syntax error will regress. ?and we make life easier for all ten writers of error recoveryfunctions, - Scanning arbitrary regions of code is not possible. ?we unleash the power of an army of grad students to studybidirectional parsing of module files, Remember: In an IDE code with syntax errors is the norm, not an exception, as the IDE provides functionality to work on incomplete code. ?and ease the burdens of the thousands who must spend theirtime looking at syntax errors for their broken module files. Nope, not for me. Give me esthetics, please. Really. ? John ---- urspr?ngliche Nachricht Ende ---- From nipa at codefx.org Fri May 19 10:41:59 2017 From: nipa at codefx.org (Nicolai Parlog) Date: Fri, 19 May 2017 12:41:59 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> Message-ID: Hi! I'm too lazy to try right now but I'm pretty sure I repeatedly observed the behavior Volker describes: A module depends on an internal type and is compiled successfully thanks to --add-exports. At run time it would fail without flags and would work with --add-exports but it also works with --add-opens. so long ... Nicolai On 19.05.2017 12:24, Peter Levart wrote: > Hi Volker, > > On 05/19/2017 11:48 AM, Volker Simonis wrote: >> From my understanding, at run-time, "open" implicates "exports" >> (i.e. if a module M1 opens a package P for some other module M2 >> it also, implicitly exports P to M2). The "big kill switch" in >> both, its old and in the newly proposed form, usually only refers >> to "enabling reflective access" but doesn't explicitly mentions >> that it will also, implicitly export the respective packages. >> Also, the usage of the "kill switch" only produces warnings for >> reflective accesses which are enabled by the option itself (and >> not at the same time, explicitly allowed by --add-opens >> directives). But it doesn't warn about the simple, non-reflective >> accesses to packages which are implicitly exported by the kill >> switch as well. > > No, you got it wrong. "opens" is just a backdoor for reflection > (and MethodHandles.Lookup). It does not implicitly also mean > "exports". Neither in compile-time nor in run-time. Otherwise > everything that is declared "public" would be accessible to > reflection without the need for .setAccessible(true) - therefore > even to code running with SecurityManager enabled! > > Regards, Peter > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From rafael.wth at gmail.com Fri May 19 10:57:15 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 12:57:15 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: Hi Alan, I understand your current time constraint, I still want to point out that this is a heavily used functionality and it not being available would cause problems in basically all production environments I am aware of where the lack of this API is not uncovered before running the application in some production or production-like environment. And while JVMTI might be optional in the spec, this is not how it is treated in practice. Therefore, I want to make this point against not having JVMTI available on the default jlink image but rather to include it by default. This can always change in a future version when this issue is better understood. As said, while it is not used by many people, a large number of Java users rely on it for their and third party application as basically any Java tooling is built around this. I just verified that this would be an issue for my current client where a lack of JVMTI would result in an infinite restart loop. Best regards, Rafael 2017-05-19 12:24 GMT+02:00 Alan Bateman : > On 19/05/2017 10:20, Rafael Winterhalter wrote: > > : >> >> It is the exact purpose of a Java agent to allow the enhancement of a >> Java application that is independent of a particular Java process. >> Therefore, it should not be a decission bound to the creation or bundeling >> of the Java application if such user-site monitoring is required. Instead, >> it should always be possible to attach a Java agent for a standalone Java >> process. Also, I cannot find any comment about the Java agent functionality >> to be optional despite the mentioned change set: >> https://docs.oracle.com/javase/8/docs/api/java/lang/instrume >> nt/package-summary.html - Neither would anybody using the functionality >> expect it to be optional. I work with agents everyday and this is the first >> time I hear it. >> >> I don't have time to debate this topic just now but just on optionally, > the tool interfaces have always been optional. It might need digging to > find all the history but as I recall, in 2004 the then in-progress JSR-163 > spec was clarified to make the optionality clear. This is why the JVM TI > spec has the statement "JVM TI may not be available in all implementations > of the JavaTM virtual machine". The wording in the java.lang.instrument > package description wasn't as clear, something that came to light when > shaking out the details of Compact Profiles for Java SE 8. This lead to a > small clarification [1] that I mentioned. > > -Alan. > > [1] http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/4a4b97f7f83b > From peter.levart at gmail.com Fri May 19 10:58:39 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 12:58:39 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> Message-ID: <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> Hi Nikolai, On 05/19/2017 12:41 PM, Nicolai Parlog wrote: > Hi! > > I'm too lazy to try right now It's easy. Take this example: public class Test { public static void main(String[] args) throws Exception { Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe") .getMethod("getUnsafe") .invoke(null); } } Running this on classpath without any additional JVM options produces: Exception in thread "main" java.lang.IllegalAccessException: class Test cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @63d4e2ba at java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445) at java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436) at java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112) at java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407) at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399) at java.base/java.lang.reflect.Method.invoke(Method.java:535) at Test.main(Test.java:9) (note that the exception is thrown from Method.invoke) When run with: --add-exports java.base/jdk.internal.misc=ALL-UNNAMED ...the example works (does not throw exception), but when run with: --add-opens java.base/jdk.internal.misc=ALL-UNNAMED ...the example still works!!!! OOOPS - this is a bug!!! This is definitely a bug. If --illegal-access=permit is an alias for opening all modules to ALL-UNNAMED, then we definitely have a problem as above example can be successfully executed even with SecurityManager enabled!!! Regards, Peter > but I'm pretty sure I repeatedly observed > the behavior Volker describes: > > A module depends on an internal type and is compiled successfully > thanks to --add-exports. At run time it would fail without flags and > would work with --add-exports but it also works with --add-opens. > > so long ... Nicolai > > > > On 19.05.2017 12:24, Peter Levart wrote: >> Hi Volker, >> >> On 05/19/2017 11:48 AM, Volker Simonis wrote: >>> From my understanding, at run-time, "open" implicates "exports" >>> (i.e. if a module M1 opens a package P for some other module M2 >>> it also, implicitly exports P to M2). The "big kill switch" in >>> both, its old and in the newly proposed form, usually only refers >>> to "enabling reflective access" but doesn't explicitly mentions >>> that it will also, implicitly export the respective packages. >>> Also, the usage of the "kill switch" only produces warnings for >>> reflective accesses which are enabled by the option itself (and >>> not at the same time, explicitly allowed by --add-opens >>> directives). But it doesn't warn about the simple, non-reflective >>> accesses to packages which are implicitly exported by the kill >>> switch as well. >> No, you got it wrong. "opens" is just a backdoor for reflection >> (and MethodHandles.Lookup). It does not implicitly also mean >> "exports". Neither in compile-time nor in run-time. Otherwise >> everything that is declared "public" would be accessible to >> reflection without the need for .setAccessible(true) - therefore >> even to code running with SecurityManager enabled! >> >> Regards, Peter >> From Alan.Bateman at oracle.com Fri May 19 11:05:35 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 12:05:35 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: On 19/05/2017 10:48, Volker Simonis wrote: > : > > From my understanding, at run-time, "open" implicates "exports" (i.e. > if a module M1 opens a package P for some other module M2 it also, > implicitly exports P to M2). The "big kill switch" in both, its old > and in the newly proposed form, usually only refers to "enabling > reflective access" but doesn't explicitly mentions that it will also, > implicitly export the respective packages. Also, the usage of the > "kill switch" only produces warnings for reflective accesses which are > enabled by the option itself (and not at the same time, explicitly > allowed by --add-opens directives). But it doesn't warn about the > simple, non-reflective accesses to packages which are implicitly > exported by the kill switch as well. You are correct. If the package is opened at run-time: 1. All public members of public types in the package are accessible. This includes both static and reflective access. 2. All members of all types in the package can be reflected on when using setAccessible or privateLookupIn. The reason you don't get warnings with the static references is because access checking in the VM is very low level and we deliberately decided to avoid the invasive changes needs to report warnings here. The latest proposal does not change this, it's just too late in JDK 9 to take on this surgery. -Alan From Alan.Bateman at oracle.com Fri May 19 11:14:00 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 12:14:00 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> Message-ID: <3c15ad23-7066-3370-010a-d46e20aa1dfc@oracle.com> On 19/05/2017 11:58, Peter Levart wrote: > : > > ...the example works (does not throw exception), but when run with: > > --add-opens java.base/jdk.internal.misc=ALL-UNNAMED > > ...the example still works!!!! OOOPS - this is a bug!!! This is > definitely a bug. No, the behavior is correct as jdk.internal.misc.Unsafe is public and it's getUnsafe is public. It's different to sun.misc.Unsafe. However, you do have a point that opening all packages by default means that the internal Unsafe might need to be looked at again. The long term plan is to reduce the need for sun.misc.Unsafe but not by having libraries move to the internal Unsafe. BTW: For the security manager case then I assume you code fragment would need to accessClassInPackage.jdk.internal.misc permission. -Alan From Alan.Bateman at oracle.com Fri May 19 11:27:40 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 12:27:40 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> Message-ID: On 19/05/2017 11:41, Nicolai Parlog wrote: > : > > A module depends on an internal type and is compiled successfully > thanks to --add-exports. At run time it would fail without flags and > would work with --add-exports but it also works with --add-opens. > That is correct, assuming you mean a reference to a public member of a public type, and the module declares `requires $MODULE` to establish the readability (this part is ignored when discussing code on the class path because unnamed modules read all modules). -Alan From peter.levart at gmail.com Fri May 19 11:28:10 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 13:28:10 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> Message-ID: On 05/19/2017 12:58 PM, Peter Levart wrote: > > It's easy. Take this example: > > public class Test { > public static void main(String[] args) throws Exception { > Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe") > .getMethod("getUnsafe") > .invoke(null); > } > } > > Running this on classpath without any additional JVM options produces: > > Exception in thread "main" java.lang.IllegalAccessException: class > Test cannot access class jdk.internal.misc.Unsafe (in module > java.base) because module java.base does not export jdk.internal.misc > to unnamed module @63d4e2ba > at > java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445) > at > java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436) > at > java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112) > at > java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407) > at > java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399) > at java.base/java.lang.reflect.Method.invoke(Method.java:535) > at Test.main(Test.java:9) > > (note that the exception is thrown from Method.invoke) > > When run with: > > --add-exports java.base/jdk.internal.misc=ALL-UNNAMED > > ...the example works (does not throw exception), but when run with: > > --add-opens java.base/jdk.internal.misc=ALL-UNNAMED > > ...the example still works!!!! OOOPS - this is a bug!!! This is > definitely a bug. > > > If --illegal-access=permit is an alias for opening all modules to > ALL-UNNAMED, then we definitely have a problem as above example can be > successfully executed even with SecurityManager enabled!!! > ...Ok, there's still a SecurityManager.checkPackageAccess() performed when some app class "loads" the jdk.internal.misc.Unsafe class and the caller needs to have a "accessClassInPackage.jdk.internal.misc" runtime permission. So this throws exception with SecurityManager enabled (from Class.forName("...") this time), but this permission should not be sufficient. I think "opens" should not imply "exports" - not even for reflection. Conceptually "exports" should fall in the same category of access controls as "private", "package-private", "protected" and "public" modifiers on members/classes. If a member is not accessible to a caller due to access modifiers on the member or enclosing class or module lacking "exports", then such member should not be allowed to be accessed: - during compilation - during runtime by bytecode instruction - during runtime by reflection - during runtime by MethodHandles.Lookup methods Opening a package to the caller module should only allow the caller to suppress these access checks, but the caller should still be required to request this explicitly either by using AccessibleObject.setAccessible(true) when reflecting; or by using MethodHandles.Lookup.privateLookupIn() when looking up members to get MethodHandles or VarHandles. Those two methods require special "suppressAccessChecks" runtime permission. Regards, Peter From adinn at redhat.com Fri May 19 11:28:51 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 19 May 2017 12:28:51 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: <9fa19263-bde2-13e5-7653-bf76d4d3131c@redhat.com> On 19/05/17 11:24, Alan Bateman wrote: > On 19/05/2017 10:20, Rafael Winterhalter wrote: > >> : >> >> It is the exact purpose of a Java agent to allow the enhancement of a >> Java application that is independent of a particular Java process. >> Therefore, it should not be a decission bound to the creation or >> bundeling of the Java application if such user-site monitoring is >> required. Instead, it should always be possible to attach a Java agent >> for a standalone Java process. Also, I cannot find any comment about >> the Java agent functionality to be optional despite the mentioned >> change set: >> https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html >> - Neither would anybody using the functionality expect it to be >> optional. I work with agents everyday and this is the first time I >> hear it. >> > I don't have time to debate this topic just now but just on optionally, > the tool interfaces have always been optional. It might need digging to > find all the history but as I recall, in 2004 the then in-progress > JSR-163 spec was clarified to make the optionality clear. This is why > the JVM TI spec has the statement "JVM TI may not be available in all > implementations of the JavaTM virtual machine". The wording in the > java.lang.instrument package description wasn't as clear, something that > came to light when shaking out the details of Compact Profiles for Java > SE 8. This lead to a small clarification [1] that I mentioned. I don't think history is the most important guide here (whatever the history might be). If agent support was not optional in the past that is no reason to not make it optional now. To me it is not a problem that agent operation may be optional. What matters is i) it is still possible to choose to include agent support in an image ii) image builders are aware when they are making such a choice ii) image users are aware that a specific image contains agent support The latter point (or two points) are, perhaps, impacted by the provision made for packaging of various standard images such as minimal, compact, full etc. I am afraid that I am not clear to what degree these 'standard' configurations are actually defined by the current tools or are simply places on the image configuration map that users of jlink must visit under their own steam (I remember reading about them in the JSR docs but I am not very familiar with the current operation or implementation of the jlink tool) Does the tooling define specific image configuratons? Is there an opportunity/need here to define and make visible a standard image format ('instrumentable') which comprises minimal + java.instrument? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Fri May 19 11:28:58 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 19 May 2017 12:28:58 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: On 19/05/17 09:22, Alan Bateman wrote: > As I said, the goal has always been to allow someone create a run-time > image that only contains java.base. I'm not sure that subsuming > java.instrument into java.base is right. Introducing options to ask > jlink to exclude modules creates the potential for conflict on the > command line. One thing that jlink could do is emit a warning that the > resulting run-time image doesn't have the management and instrumentation > features, might that be the right balance. Like Remi, I think this is an entirely reasonable position. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From peter.levart at gmail.com Fri May 19 11:40:07 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 13:40:07 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <3c15ad23-7066-3370-010a-d46e20aa1dfc@oracle.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> <3c15ad23-7066-3370-010a-d46e20aa1dfc@oracle.com> Message-ID: <92616307-503e-f337-5606-1ee7f08509aa@gmail.com> On 05/19/2017 01:14 PM, Alan Bateman wrote: > On 19/05/2017 11:58, Peter Levart wrote: > >> : >> >> ...the example works (does not throw exception), but when run with: >> >> --add-opens java.base/jdk.internal.misc=ALL-UNNAMED >> >> ...the example still works!!!! OOOPS - this is a bug!!! This is >> definitely a bug. > No, the behavior is correct as jdk.internal.misc.Unsafe is public and > it's getUnsafe is public. It's different to sun.misc.Unsafe. > > However, you do have a point that opening all packages by default > means that the internal Unsafe might need to be looked at again. The > long term plan is to reduce the need for sun.misc.Unsafe but not by > having libraries move to the internal Unsafe. > > BTW: For the security manager case then I assume you code fragment > would need to accessClassInPackage.jdk.internal.misc permission. Right, but such permission should not be sufficient, since it is required only to obtain the jdk.internal.misc.Unsafe.class object (i.e. for Class.forName("jdk.internal.misc.Unsafe"). I can play a trick here to obtain such Class instance. For example: public class Test { public static void main(String[] args) throws Exception { System.setSecurityManager(new SecurityManager()); Class unsafeClass = java.io.File.class.getDeclaredField("UNSAFE").getType(); Object theUnsafe = unsafeClass.getMethod("getUnsafe").invoke(null); } } Ok, I need "accessDeclaredMembers" permission here, but you get the picture... Regards, Peter > > -Alan From adinn at redhat.com Fri May 19 11:41:01 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 19 May 2017 12:41:01 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: <54f94db4-c34d-0831-43b1-3094b6ff1a2c@redhat.com> HI Rafael, On 19/05/17 11:57, Rafael Winterhalter wrote: > Hi Alan, > I understand your current time constraint, I still want to point out that > this is a heavily used functionality and it not being available would cause > problems in basically all production environments I am aware of where the > lack of this API is not uncovered before running the application in some > production or production-like environment. And while JVMTI might be > optional in the spec, this is not how it is treated in practice. Therefore, > I want to make this point against not having JVMTI available on the default > jlink image but rather to include it by default. This can always change in > a future version when this issue is better understood. As said, while it is > not used by many people, a large number of Java users rely on it for their > and third party application as basically any Java tooling is built around > this. I just verified that this would be an issue for my current client > where a lack of JVMTI would result in an infinite restart loop. > Best regards, Rafael I think you have the cart before the horse here. This is only going to affect those who deploy apps which use custom images. It is not going to affect anyone using a standard JDK release image. For all practical purposes this is unlikely to be a common circumstance. Your desire to stop people painting themselves into a corner is, no doubt, morally commendable. However, it is an error from which those affected will be able to obtain remedy without your help (by fixing their image). Personally, I think it would be better to leave anyone who puts themselves in such an unhappy situation to implement the solution for themselves. I will, of course, still do my best to educate all Byteman users about the pitfalls that might accompany careless drawing up of interior decor procedures. I don't really see why the JDK team need to do that job for me. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From peter.levart at gmail.com Fri May 19 12:02:38 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 14:02:38 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> Hi Alan, On 05/19/2017 01:05 PM, Alan Bateman wrote: > On 19/05/2017 10:48, Volker Simonis wrote: > >> : >> >> From my understanding, at run-time, "open" implicates "exports" (i.e. >> if a module M1 opens a package P for some other module M2 it also, >> implicitly exports P to M2). The "big kill switch" in both, its old >> and in the newly proposed form, usually only refers to "enabling >> reflective access" but doesn't explicitly mentions that it will also, >> implicitly export the respective packages. Also, the usage of the >> "kill switch" only produces warnings for reflective accesses which are >> enabled by the option itself (and not at the same time, explicitly >> allowed by --add-opens directives). But it doesn't warn about the >> simple, non-reflective accesses to packages which are implicitly >> exported by the kill switch as well. > You are correct. If the package is opened at run-time: > > 1. All public members of public types in the package are accessible. > This includes both static and reflective access. > > 2. All members of all types in the package can be reflected on when > using setAccessible or privateLookupIn. > > The reason you don't get warnings with the static references is > because access checking in the VM is very low level and we > deliberately decided to avoid the invasive changes needs to report > warnings here. The latest proposal does not change this, it's just too > late in JDK 9 to take on this surgery. > > -Alan Ok, I can understand that such interpretation of "opens" is necessary for JDK 8 compatibility story. If something is "open" then it behaves exactly like in JDK 8 or before (apart from compilation) regardless of packages being exported or not. But from consistency perspective, explicit "exports" should be required just like "public" modifiers are required to access members without .setAccessible(true) or without .privateLookupIn(). This would not hinder the ability for frameworks to access members. It would just require them to call .setAccessible(true)/.privateLookupIn() in more places, but would be safer too as "suppressAccessChecks" permission would be required for non-exported package access like it is required for "private" members. I'm sorry I haven't noticed this before. It is probably to late to change this now. Was this deliberate choice (to promote compatibility with existing code)? Regards, Peter From rafael.wth at gmail.com Fri May 19 12:08:25 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 14:08:25 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <54f94db4-c34d-0831-43b1-3094b6ff1a2c@redhat.com> References: <54f94db4-c34d-0831-43b1-3094b6ff1a2c@redhat.com> Message-ID: Hi Andrew, do not forget that we are talking about 130 kilobyte here which come at the costs of removing a feature that often serves as a last resort and where its need is rarely anticipated. Given the little overhead of including this module, I do not find it reasonable to disable a feature that aims to making cross-cutting functionality such as monitoring or production-level debugging detachable from a Java program. When working with customers, changing a product pipe-line might be technically trivial but still involves meetings, waitings and politics where every step costs money and breaks deadlines. In this context, just having the Java agent functionality working is a big advantage and I find it sad to sacrifice this property over so little gain. Finally, there is a chance that a Java application vendor deliberately removes Java agent support to make a product unmonitorable for generic tools in order to improve their position for selling a commercial tool based on a proprietary API. This would of course improve their position but clutter the Java agent system for operation teams. Java is mostly used in the enterprise and Java's universal runnability is an important property. Having agents not run on most jlink images does not improve this position. Best regards, Rafael 2017-05-19 13:41 GMT+02:00 Andrew Dinn : > HI Rafael, > > On 19/05/17 11:57, Rafael Winterhalter wrote: > > Hi Alan, > > I understand your current time constraint, I still want to point out that > > this is a heavily used functionality and it not being available would > cause > > problems in basically all production environments I am aware of where the > > lack of this API is not uncovered before running the application in some > > production or production-like environment. And while JVMTI might be > > optional in the spec, this is not how it is treated in practice. > Therefore, > > I want to make this point against not having JVMTI available on the > default > > jlink image but rather to include it by default. This can always change > in > > a future version when this issue is better understood. As said, while it > is > > not used by many people, a large number of Java users rely on it for > their > > and third party application as basically any Java tooling is built around > > this. I just verified that this would be an issue for my current client > > where a lack of JVMTI would result in an infinite restart loop. > > Best regards, Rafael > > I think you have the cart before the horse here. This is only going to > affect those who deploy apps which use custom images. It is not going to > affect anyone using a standard JDK release image. For all practical > purposes this is unlikely to be a common circumstance. > > Your desire to stop people painting themselves into a corner is, no > doubt, morally commendable. However, it is an error from which those > affected will be able to obtain remedy without your help (by fixing > their image). Personally, I think it would be better to leave anyone who > puts themselves in such an unhappy situation to implement the solution > for themselves. I will, of course, still do my best to educate all > Byteman users about the pitfalls that might accompany careless drawing > up of interior decor procedures. I don't really see why the JDK team > need to do that job for me. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From david.lloyd at redhat.com Fri May 19 12:10:35 2017 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 19 May 2017 07:10:35 -0500 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <9a562c08-beb3-f7fb-23cf-faaf13cecd7e@gmail.com> <20c035e5-c65f-c30f-d029-b3a421cd23d2@gmail.com> Message-ID: On 05/19/2017 05:58 AM, Peter Levart wrote: > Hi Nikolai, > > On 05/19/2017 12:41 PM, Nicolai Parlog wrote: >> Hi! >> >> I'm too lazy to try right now > > It's easy. Take this example: > > public class Test { > public static void main(String[] args) throws Exception { > Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe") > .getMethod("getUnsafe") > .invoke(null); > } > } > > Running this on classpath without any additional JVM options produces: > > Exception in thread "main" java.lang.IllegalAccessException: class Test > cannot access class jdk.internal.misc.Unsafe (in module java.base) > because module java.base does not export jdk.internal.misc to unnamed > module @63d4e2ba > at > java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445) > > at > java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436) > > at > java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112) > > at > java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407) > > at > java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399) > > at java.base/java.lang.reflect.Method.invoke(Method.java:535) > at Test.main(Test.java:9) > > (note that the exception is thrown from Method.invoke) > > When run with: > > --add-exports java.base/jdk.internal.misc=ALL-UNNAMED > > ...the example works (does not throw exception), but when run with: > > --add-opens java.base/jdk.internal.misc=ALL-UNNAMED > > ...the example still works!!!! OOOPS - this is a bug!!! This is > definitely a bug. This is seemingly consistent with the code run time code in Module for addOpens, which always seems to add export when open is added. It would be quite strange if the descriptor directives and JVM switches didn't match the run time code; is this something that the EG should talk about? Or is there a hidden logic to this? -- - DML From adinn at redhat.com Fri May 19 12:22:44 2017 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 19 May 2017 13:22:44 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: <54f94db4-c34d-0831-43b1-3094b6ff1a2c@redhat.com> Message-ID: <2a68dd28-deba-85d6-427c-1ee2425e4b15@redhat.com> Hi Rafael, On 19/05/17 13:08, Rafael Winterhalter wrote: > do not forget that we are talking about 130 kilobyte here which come at > the costs of removing a feature that often serves as a last resort and > where its need is rarely anticipated. Given the little overhead of > including this module, I do not find it reasonable to disable a feature > that aims to making cross-cutting functionality such as monitoring or > production-level debugging detachable from a Java program. I am aware of what little there is to be saved here. However, what you find reasonable others may find unreasonable. So,I am glad that omitting the agent support is an option for those who want it. > When working with customers, changing a product pipe-line might be > technically trivial but still involves meetings, waitings and politics > where every step costs money and breaks deadlines. In this context, just > having the Java agent functionality working is a big advantage and I > find it sad to sacrifice this property over so little gain. I understand this too s do our support team. I will do my best to ensure that Red Hat's own products include agent support in the images we provide and to ensure that Red Hat makes customers aware of the benefits of including agent support in any images they produce and distribute. However, I don't see the need to strong arm anyone here (see below). > Finally, there is a chance that a Java application vendor deliberately > removes Java agent support to make a product unmonitorable for generic > tools in order to improve their position for selling a commercial tool > based on a proprietary API. This would of course improve their position > but clutter the Java agent system for operation teams. Java is mostly > used in the enterprise and Java's universal runnability is an important > property. Having agents not run on most jlink images does not improve > this position. Well, your first proposition is very easy to dismiss as FUD. Do you have any reason to fear that unscrupulous vendors will try to do what you suggest? And if they do is that not merely a reason for Java devs/users to resort to another vendor who doesn't behave so unscrupulously. Also, how would it benefit any vendor to cripple their own ability to support their product? Also, you say blithely hypothesise "having agents not run on most jlink images" as though this is already a fait accompli. What makes you assume that the mere /possibility/ of not including agent support will translate to a high number of deployments opting for that possibility? If agents are as important as you say (and I think they probably are) then such a move would be as foolish as it is unlikely. Sorry but this argument looks like FUD too. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From Alan.Bateman at oracle.com Fri May 19 12:28:48 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 13:28:48 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> Message-ID: <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> On 19/05/2017 13:02, Peter Levart wrote: > > Ok, I can understand that such interpretation of "opens" is necessary > for JDK 8 compatibility story. If something is "open" then it behaves > exactly like in JDK 8 or before (apart from compilation) regardless of > packages being exported or not. But from consistency perspective, > explicit "exports" should be required just like "public" modifiers are > required to access members without .setAccessible(true) or without > .privateLookupIn(). > > This would not hinder the ability for frameworks to access members. It > would just require them to call > .setAccessible(true)/.privateLookupIn() in more places, but would be > safer too as "suppressAccessChecks" permission would be required for > non-exported package access like it is required for "private" members. > > I'm sorry I haven't noticed this before. It is probably to late to > change this now. Was this deliberate choice (to promote compatibility > with existing code)? `opens` is intended to open a package for reflective access. Code can use the reflection APIs to get at public members of public types in the package, it shouldn't need setAccessible to do that. The static reference case that Volker brought up is somewhat of a corner case in this discussion. Readability aside, it would be inconsistent if reflection APIs could access public members of public types in the package but bytecode could not. The other corner case is code generation at run-time which also should also work (assuming both visibility and readability). -Alan From aph at redhat.com Fri May 19 12:54:33 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 19 May 2017 13:54:33 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> Message-ID: <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> On 19/05/17 11:11, Peter Levart wrote: > On 05/19/2017 01:17 AM, Nicolai Parlog wrote: >> With illegal access being permitted by default much fewer developers >> will be aware of the problem and much less pressure will be put on >> library and framework maintainers as well as on project management to >> invest into paying back this particular form of technical debt. So we >> get much less momentum to make the necessary changes in exchange for... >> not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the >> Java community! > > +1 > > I think that --illegal-access=permit-silently or equivalent is a > reasonable request (in addition to other modes), but please make the > --illegal-access=deny the default! I don't think that's practical because, as Mark Reinhold put it, "the strong encapsulation of JDK-internal APIs has, in particular, triggered many worried expressions of concern that code that works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of this change was given in JDK 8." Andrew. From Alan.Bateman at oracle.com Fri May 19 13:03:59 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 14:03:59 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <9fa19263-bde2-13e5-7653-bf76d4d3131c@redhat.com> References: <9fa19263-bde2-13e5-7653-bf76d4d3131c@redhat.com> Message-ID: <5cca12df-88e7-1e95-82f3-799c0c1230b7@oracle.com> On 19/05/2017 12:28, Andrew Dinn wrote: > : > I don't think history is the most important guide here (whatever the > history might be). If agent support was not optional in the past that is > no reason to not make it optional now. My reason for bringing up the history is to make it clear that the standard tooling APIs have always been optional ("always" = JSR-163 / Java SE 5). A clarification was needed in Java SE 8 to fix an oversight. We haven't changed anything for Java SE 9. > > To me it is not a problem that agent operation may be optional. What > matters is > > i) it is still possible to choose to include agent support in an image There are a number of pieces to this. The biggest part is the JVM TI support that is compiled into libjvm. The Oracle JDK downloads include the server VM and the minimal VM. The former has the serviceability support compiled in, the minimal VM does not. When using `jlink` then you would need to go out of your way to specify `--vm=minimal` to exclude the server VM. For java agent support then you need a java.base/libjvm that includes JVM TI and you additionally need to ask `jlink` to include the `java.instrument` module. There are several improvements that could be done here to ensure that you don't accidentally create a run-time image with java.instrument but no libjvm with JVM TI support, ditto for the management modules that need instrumentation in the VM. Another part is the java agent itself. It would be be nice to be able to deploy a java agent as a module so that it can be specified to `jlink`. We had to punt on this for Java SE 9 / JDK 9 but it's definitely something to come back to in the future (only so much that we could take on here in one release). > : > > Does the tooling define specific image configuratons? Is there an > opportunity/need here to define and make visible a standard image format > ('instrumentable') which comprises minimal + java.instrument? > Do you mean an aggregator module? Possible although I think `jlink` usages are better suited to recipes, esp. when getting into some of the trade-offs. -Alan From rafael.wth at gmail.com Fri May 19 13:30:25 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 15:30:25 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <2a68dd28-deba-85d6-427c-1ee2425e4b15@redhat.com> References: <54f94db4-c34d-0831-43b1-3094b6ff1a2c@redhat.com> <2a68dd28-deba-85d6-427c-1ee2425e4b15@redhat.com> Message-ID: Most project teams I come accross as a consultant broadly ignore warnings. I would not expect that most people pay attention to a warning that an image is not including JVMTI, especially if the implications are unclear to the person in charge of running the build. Excluding JVMTI by default - ie excluding JVMTI from most jlink images as I said - is therefore not a good default in my opinion. As for your "this is FUD" argument, I can say that I have been in situations like I described myself. The mentioned problem does not necessarily mean that a vendor would deliberatly change their product to break a competitor but there is a good chance that they will not respond to a request to include JVMTI in their image if they do not require it themselves. Especially if this would enable a supplementary product to compete with a (paid) feature of an open core platform's maintainer. Personally, I did however even experience the first scenario. Obviously, I cannot name you details due to signed NDAs and it is up to you if you want to believe me. But if you need a specific example of this behavior, look no further than the Apple app store and their history of removing applications that compete with Apple's own products. Again, I do not say that this will happen all of the time or even often, but I expect it to happen sometimes. Therefore, I find it reasonable to include JVMTI on any jlink image where it comes at the cost of 130 kilobyte. 2017-05-19 14:22 GMT+02:00 Andrew Dinn : > Hi Rafael, > > On 19/05/17 13:08, Rafael Winterhalter wrote: > > do not forget that we are talking about 130 kilobyte here which come at > > the costs of removing a feature that often serves as a last resort and > > where its need is rarely anticipated. Given the little overhead of > > including this module, I do not find it reasonable to disable a feature > > that aims to making cross-cutting functionality such as monitoring or > > production-level debugging detachable from a Java program. > > I am aware of what little there is to be saved here. However, what you > find reasonable others may find unreasonable. So,I am glad that omitting > the agent support is an option for those who want it. > > > When working with customers, changing a product pipe-line might be > > technically trivial but still involves meetings, waitings and politics > > where every step costs money and breaks deadlines. In this context, just > > having the Java agent functionality working is a big advantage and I > > find it sad to sacrifice this property over so little gain. > > I understand this too s do our support team. I will do my best to ensure > that Red Hat's own products include agent support in the images we > provide and to ensure that Red Hat makes customers aware of the benefits > of including agent support in any images they produce and distribute. > However, I don't see the need to strong arm anyone here (see below). > > > Finally, there is a chance that a Java application vendor deliberately > > removes Java agent support to make a product unmonitorable for generic > > tools in order to improve their position for selling a commercial tool > > based on a proprietary API. This would of course improve their position > > but clutter the Java agent system for operation teams. Java is mostly > > used in the enterprise and Java's universal runnability is an important > > property. Having agents not run on most jlink images does not improve > > this position. > > Well, your first proposition is very easy to dismiss as FUD. Do you have > any reason to fear that unscrupulous vendors will try to do what you > suggest? And if they do is that not merely a reason for Java devs/users > to resort to another vendor who doesn't behave so unscrupulously. > > Also, how would it benefit any vendor to cripple their own ability to > support their product? > > Also, you say blithely hypothesise "having agents not run on most jlink > images" as though this is already a fait accompli. What makes you assume > that the mere /possibility/ of not including agent support will > translate to a high number of deployments opting for that possibility? > If agents are as important as you say (and I think they probably are) > then such a move would be as foolish as it is unlikely. Sorry but this > argument looks like FUD too. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From stephen.felts at oracle.com Fri May 19 13:38:27 2017 From: stephen.felts at oracle.com (Stephen Felts) Date: Fri, 19 May 2017 06:38:27 -0700 (PDT) Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> Message-ID: <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> Here's something to think about with respect to noise by whatever choice is made for the default for --illegal-access. ? I've run a lot of code using the current --permit-illegal-access. ? One of the problems with this option is that there can be errors detected for code that is working correctly and/or has already been fixed for JDK9. Since I have been working on identifying and entering bugs for JDK9 problems, I have also been collecting a list of warnings for code that is valid so I can ignore them. Here are a few public ones. org.python.core.PyJavaClass - Jython com.sun.xml.bind.v2.runtime.reflect.opt.Injector ? standalone jaxb com.sun.xml.ws.model.Injector ? standalone jaxws net.sf.cglib.core.ReflectUtils$ - cglib ? Summarizing the use cases: 1. The code calls setAccessible and correctly catches and ignores the exception for those that fail.? It would be a big change to make this call setAccessible lazily. 2. The code continues to use the JDK8 approach and if that fails it falls back to an approach that works on JDK9 and/or other non-Hotspot platforms.? In some cases, this code has been around for a decade or more (e.g., to handle platforms without Unsafe). ? In work flows that I have seen, a message would always be printed for every process, even though there is no real problem.? That?s not acceptable in a production environment. ? Let's assume that there is at least one illegal access per process that is run. The current proposal will print one message per process.? That can be a lot of noise.? Examples: - Running tests with fork for every test (obviously not a production environment). - If you are doing internal process generation that used to be invisible to your customers, it's now visible. ? ? From nipa at codefx.org Fri May 19 13:39:36 2017 From: nipa at codefx.org (Nicolai Parlog) Date: Fri, 19 May 2017 15:39:36 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> Message-ID: <48ac04cf-36db-283e-0de6-522d94ef0b07@codefx.org> Hi, I don't want to pick on Andrew in particular but his paragraph is a great summary of the sad state of this discussion: > I don't think that's practical Interesting, waiting for a real, practical, technical reason not to do that. > many worried expressions Err, what? So something is impractical because people are worried? I'm sure they're worried because they think it's impractical. > code that works on JDK 8 today will not work on JDK 9 tomorrow, > yet no advance warning of this change was given in JDK 8 So twenty years of claiming internal APIs are internal and subject to change[1] was not enough? Two years of this change being announced was no enough? One more year of Java 8 is not enough? A simple flag to make it work is not enough? so long ... Nicolai [1] http://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html On 19.05.2017 14:54, Andrew Haley wrote: > On 19/05/17 11:11, Peter Levart wrote: >> On 05/19/2017 01:17 AM, Nicolai Parlog wrote: >>> With illegal access being permitted by default much fewer developers >>> will be aware of the problem and much less pressure will be put on >>> library and framework maintainers as well as on project management to >>> invest into paying back this particular form of technical debt. So we >>> get much less momentum to make the necessary changes in exchange for... >>> not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the >>> Java community! >> >> +1 >> >> I think that --illegal-access=permit-silently or equivalent is a >> reasonable request (in addition to other modes), but please make the >> --illegal-access=deny the default! > > I don't think that's practical because, as Mark Reinhold put it, "the > strong encapsulation of JDK-internal APIs has, in particular, > triggered many worried expressions of concern that code that works on > JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of > this change was given in JDK 8." > > Andrew. > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From stephen.felts at oracle.com Fri May 19 13:48:29 2017 From: stephen.felts at oracle.com (Stephen Felts) Date: Fri, 19 May 2017 06:48:29 -0700 (PDT) Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> Message-ID: By the way, there is no available API so that one can write ? If an object is not accessible and setAccessible on the object will succeed ? call setAccessible ? That means that the code needs to call setAccessible and handle the exception, which triggers a warning. ? ? ? From forax at univ-mlv.fr Fri May 19 13:53:47 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 19 May 2017 15:53:47 +0200 (CEST) Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: <1495190227.645288-11404@martha.daybyday.de> References: <1495190227.645288-11404@martha.daybyday.de> Message-ID: <229221312.3399996.1495202027081.JavaMail.zimbra@u-pem.fr> > De: "Stephan Herrmann" > ?: "John Rose" , jigsaw-dev at openjdk.java.net > Cc: "R?mi Forax" > Envoy?: Vendredi 19 Mai 2017 12:37:07 > Objet: Re: Re: An alternative to "restricted keywords" + helping automatic > modules > A quick question to keep the ball rolling: > Do we agree on the following assessment of the status quo? > The definition of "restricted keywords" implies (without explicitly saying so), > that classification of a word as keyword vs. identifier can only be made > *after* parsing has accepted the enclosing ModuleDeclaration. > (With some tweaks, this can be narrowed down to > "after the enclosing ModuleDirective has been accepted") > This definition is not acceptable. I agree that this is not acceptable but this is not what we are proposing. You do not have to wait the reduction of ModuleDeclaration (or ModuleDirective), the parser know its parsing state (the LR item) during the parsing not at the end. The LR analysis is not able to know at some point during the parsing which production will be reduced later but it is able to know which terminals will not lead to an error when shifting the next terminal. When you are in the middle of the parsing, the parser shift a terminal to go from one state to another, so for a state the parser knows if it can shift by a terminal which is among the set of restricted keywords or not then either it can instruct the lexer before scanning the token to activate the restricted keyword automata or after having scanned the token it can classify the token as a keyword instead of as an identifier. The idea is that the parser will not only tell when it reduces a production but also when it is about to shift a restricted keyword. So you can classify a token as an identifier or as a keyword because the parser is able to bubble up that its parser state (the LR item) may recognize a keyword. > comments? > Stephan R?mi > ----- urspr?ngliche Nachricht --------- > Subject: Re: An alternative to "restricted keywords" + helping automatic modules > Date: Fr 19 Mai 2017 07:27:31 CEST > From: John Rose > To: Stephan Herrmann > On May 18, 2017, at 1:59 AM, Stephan Herrmann < stephan.herrmann at berlin.de > > wrote: >> In all posts I could not find a real reason against escaping, >> aside from aesthetics. I don't see this as sufficient motivation >> for a less-then-perfect solution. > So, by disregarding esthetics... >> Clarity: >> I'm still not completely following your explanations, partly because >> of the jargon you are using. I'll leave it to Alex to decide if he >> likes the idea that JLS would have to explain terms like dotted >> production. >> Compare this to just adding a few more rules to the grammar, >> where no hand-waving is needed for an explanation. >> No, I did not say that escaping is a pervasive change. >> I never said that the grammar for ordinary compilation units >> should be changed. >> If you like we only need to extend one rule for the scope of >> modular compilation units: Identifier. It can't get simpler. >> Completeness: >> I understand you as saying, module names cannot start with >> "transitive". Mind you, that every modifier that will be added >> to the grammar for modules in the future will cause conflicts for >> names that are now legal, and you won't have a means to resolve this. >> By contrast, we can use the escaping approach even to solve one >> more problem that has been briefly touched on this list before: >> Automatic modules suffer from the fact that some artifact names may >> have Java keywords in their name, which means that these artifacts >> simply cannot be used as automatic modules, right? >> Why not apply escaping also here? *Any* dot-separated sequence >> of words could be used as module name, as long as module references >> have a means to escape any keywords in that sequence. >> Suitability for implementation: >> As said, your proposal resolves one problem, but still IDE >> functionality suffers from restricted keywords, because scanning >> and parsing need more context information than normal. > ?we obtain the freedom for IDEs to disregard abnormal > amounts of context, saving uncounted machine cycles, >> - Recovery after a syntax error will regress. > ?and we make life easier for all ten writers of error recovery > functions, >> - Scanning arbitrary regions of code is not possible. > ?we unleash the power of an army of grad students to study > bidirectional parsing of module files, >> Remember: >> In an IDE code with syntax errors is the norm, not an exception, >> as the IDE provides functionality to work on incomplete code. > ?and ease the burdens of the thousands who must spend their > time looking at syntax errors for their broken module files. > Nope, not for me. Give me esthetics, please. Really. > ? John > ---- urspr?ngliche Nachricht Ende ---- From peter.levart at gmail.com Fri May 19 13:54:19 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 15:54:19 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> Message-ID: Hi Alan, On 05/19/2017 02:28 PM, Alan Bateman wrote: > On 19/05/2017 13:02, Peter Levart wrote: > >> >> Ok, I can understand that such interpretation of "opens" is necessary >> for JDK 8 compatibility story. If something is "open" then it behaves >> exactly like in JDK 8 or before (apart from compilation) regardless >> of packages being exported or not. But from consistency perspective, >> explicit "exports" should be required just like "public" modifiers >> are required to access members without .setAccessible(true) or >> without .privateLookupIn(). >> >> This would not hinder the ability for frameworks to access members. >> It would just require them to call >> .setAccessible(true)/.privateLookupIn() in more places, but would be >> safer too as "suppressAccessChecks" permission would be required for >> non-exported package access like it is required for "private" members. >> >> I'm sorry I haven't noticed this before. It is probably to late to >> change this now. Was this deliberate choice (to promote compatibility >> with existing code)? > `opens` is intended to open a package for reflective access. Code can > use the reflection APIs to get at public members of public types in > the package, it shouldn't need setAccessible to do that. Then "opens" is a sharp tool. And the proposal is about applying this tool to the whole JDK by default! Why I think this is not good? Because "opens", as it is, caries two aspects or purposes at the same time. One is the purpose of giving access to frameworks that introspect classes of a module and also access them (DI frameworks for example) and this is a desirable purpose. The other is the undesirable consequence of giving "unrestricted" access to public types in the opened package during runtime. Introspective access should be paired with a runtime permission in a secured system while "unrestricted" access should only be given explicitly - using good old "exports". I see the problem with that. "opens" and "exports" are applied to packages, but "open" can also be applied to the whole module and we don't have an equivalent for "exports" on the module-level. But is that really needed? Introspective frameworks should not have a problem with invoking .setAccessible() as they could do that mechanically. Opening the whole JDK (--illegal-access=permit by default) means that all internal "public" APIs are made accessible if by chance someone can grab an instance of target object and/or an instance of Method/Field object. Imagine a JDK developer that thought that by putting a public type into a concealed package was equivalent to making the type module-private. This is a big surprise from the security perspective and jdk.internal.misc.Unsafe.getUnsafe() might not be a lone example. > > The static reference case that Volker brought up is somewhat of a > corner case in this discussion. Readability aside, it would be > inconsistent if reflection APIs could access public members of public > types in the package but bytecode could not. The other corner case is > code generation at run-time which also should also work (assuming both > visibility and readability). I get that. Generated code would have less access than reflection and generated code is often used to replace reflection to speed things up. Huh, not an easy choice, I agree... Regards, Peter > > -Alan From Alan.Bateman at oracle.com Fri May 19 14:05:40 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 15:05:40 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> Message-ID: <51eb8bfb-19f6-fbd3-bf5b-7effae32611f@oracle.com> On 19/05/2017 14:54, Peter Levart wrote: > : > > Opening the whole JDK (--illegal-access=permit by default) means that > all internal "public" APIs are made accessible if by chance someone > can grab an instance of target object and/or an instance of > Method/Field object. Imagine a JDK developer that thought that by > putting a public type into a concealed package was equivalent to > making the type module-private. This is a big surprise from the > security perspective and jdk.internal.misc.Unsafe.getUnsafe() might > not be a lone example. True although it's no different to JDK 8 and older behavior where all public members of all public types in all packages were accessible to code on the class path. Furthermore, setAccessible could be use to hack everywhere. The proposal is really just giving libraries and tools more time to sort out their issues. -Alan From russell.gold at oracle.com Fri May 19 14:08:32 2017 From: russell.gold at oracle.com (Russell Gold) Date: Fri, 19 May 2017 10:08:32 -0400 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <48ac04cf-36db-283e-0de6-522d94ef0b07@codefx.org> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> <48ac04cf-36db-283e-0de6-522d94ef0b07@codefx.org> Message-ID: <9759A941-269E-48DE-9C90-3B0D0F96D1A4@oracle.com> There are two separate issues, here: 1) Access to internal APIs. Yes, we have been warned of these for two decades; it shouldn?t actually be a shock that they have been banned. It shouldn?t, but in some cases, people didn?t really believe that they would go away. Still, it seems reasonable to say that anything marked with a deprecation warning for some time can vanish without further notice. That is the point of them, after all. 2) Reflective access to internals. This one isn?t so easy. Obviously, when you are using reflection to hack into somebody else?s code, you should expect that it could change without warning; however, this has always seemed to work, and people have come to rely on it. In many cases, it has allowed Java developers to do things that were not possible in other languages. Because it is runtime functionality, there were no equivalents of the deprecation warnings. This new proposal creates them, giving notice that we need to find alternatives. As I see it, the ideal default is compilation failures against internal APIs by default, but access with warnings of to-be-removed reflective access to internals. Command-line switches for runtime is challenging, since many Java programs are run from user-written shell scripts (the executable jar case is handled by manifest entries). Command-line switches for compilation shouldn?t be that much of a burden. Some of us have been working to remove all of our internal accesses, but twenty-years of technical debt isn?t easily cleaned up in a short time - especially when some it relies on third-party code. This proposal allows us to move forward. It makes it abundantly clear that things are changing: deprecated and internal APIs are really going away, and reflective access is going to be limited. At the same time, it allows the vast amount of legacy code out there to migrate to the new world sooner rather than later. > On May 19, 2017, at 9:39 AM, Nicolai Parlog wrote: > > Hi, > > I don't want to pick on Andrew in particular but his paragraph is a > great summary of the sad state of this discussion: > >> I don't think that's practical > > Interesting, waiting for a real, practical, technical reason not to do that. > >> many worried expressions > > Err, what? So something is impractical because people are worried? I'm > sure they're worried because they think it's impractical. > >> code that works on JDK 8 today will not work on JDK 9 tomorrow, >> yet no advance warning of this change was given in JDK 8 > > So twenty years of claiming internal APIs are internal and subject to > change[1] was not enough? Two years of this change being announced was > no enough? One more year of Java 8 is not enough? A simple flag to make > it work is not enough? > > so long ... Nicolai > > > [1] > http://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html > > > > On 19.05.2017 14:54, Andrew Haley wrote: >> On 19/05/17 11:11, Peter Levart wrote: >>> On 05/19/2017 01:17 AM, Nicolai Parlog wrote: >>>> With illegal access being permitted by default much fewer developers >>>> will be aware of the problem and much less pressure will be put on >>>> library and framework maintainers as well as on project management to >>>> invest into paying back this particular form of technical debt. So we >>>> get much less momentum to make the necessary changes in exchange for... >>>> not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the >>>> Java community! >>> >>> +1 >>> >>> I think that --illegal-access=permit-silently or equivalent is a >>> reasonable request (in addition to other modes), but please make the >>> --illegal-access=deny the default! >> >> I don't think that's practical because, as Mark Reinhold put it, "the >> strong encapsulation of JDK-internal APIs has, in particular, >> triggered many worried expressions of concern that code that works on >> JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of >> this change was given in JDK 8." >> >> Andrew. >> > > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > https://www.sitepoint.com/java > high-quality Java/JVM content > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx From Alan.Bateman at oracle.com Fri May 19 14:11:17 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 15:11:17 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> Message-ID: On 19/05/2017 14:48, Stephen Felts wrote: > By the way, there is no available API so that one can write > > > > If an object is not accessible and setAccessible on the object will succeed > > call setAccessible > > > > That means that the code needs to call setAccessible and handle the exception, which triggers a warning. > > AccessibleObject canAccess is new in 9 to test if a reflected object is accessible. Another new API is trySetAccessible to avoid using exceptions for control flow. To your example then if setAccessible succeeds in suppressing access to a JDK internal member then you'll get a warning if it's the first illegal access. If setAccessible fails then you'll InaccessibleObjectException will be thrown as it does not, there will be no warning. -Alan. From peter.levart at gmail.com Fri May 19 14:27:53 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 16:27:53 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <51eb8bfb-19f6-fbd3-bf5b-7effae32611f@oracle.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> <51eb8bfb-19f6-fbd3-bf5b-7effae32611f@oracle.com> Message-ID: <6ae79561-cbfd-5d2a-a171-66007999ea66@gmail.com> On 05/19/2017 04:05 PM, Alan Bateman wrote: > On 19/05/2017 14:54, Peter Levart wrote: > >> : >> >> Opening the whole JDK (--illegal-access=permit by default) means that >> all internal "public" APIs are made accessible if by chance someone >> can grab an instance of target object and/or an instance of >> Method/Field object. Imagine a JDK developer that thought that by >> putting a public type into a concealed package was equivalent to >> making the type module-private. This is a big surprise from the >> security perspective and jdk.internal.misc.Unsafe.getUnsafe() might >> not be a lone example. > True although it's no different to JDK 8 and older behavior where all > public members of all public types in all packages were accessible to > code on the class path. Except that in the meanwhile a lot of internal code was written for JDK 9 that assumed the level of privacy provided by concealed packages. This level is about to be changed by the proposal in the last minute... > Furthermore, setAccessible could be use to hack everywhere. The > proposal is really just giving libraries and tools more time to sort > out their issues. > > -Alan Regards, Peter From michael.rasmussen at zeroturnaround.com Fri May 19 15:07:14 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Fri, 19 May 2017 18:07:14 +0300 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: On 19 May 2017 at 11:22, Alan Bateman wrote: > One thing that jlink > could do is emit a warning that the resulting run-time image doesn't have > the management and instrumentation features, might that be the right > balance. As a users of those kind of agents, and as an agent vendor myself (though not one I expect to be used often with jlink'ed images - but how app servers in the future are distributed remains to be seen) I get where Rafael is coming from. I also agree it will come to a surprise to many, that if suddenly the distributed image from a vendor, now created by jlink, no longer included the capability of attaching agents! Having serviceability/agent support included by default makes sense to me, but I also get the reverse argument, that you should be able to create a minimalvm/java.base image, if so desired. I most definitely think, that jlink should at least emit a warning, if the image it's generating does not include those features, and also have the warning include information how to add these features. A dedicated option for jlink to explicitly enable/disable serviceability/agent modules could be nice for that. /Michael From stephan.herrmann at berlin.de Fri May 19 15:26:02 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 19 May 2017 17:26:02 +0200 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: <229221312.3399996.1495202027081.JavaMail.zimbra@u-pem.fr> References: <1495190227.645288-11404@martha.daybyday.de> <229221312.3399996.1495202027081.JavaMail.zimbra@u-pem.fr> Message-ID: <10567fd5-6776-8beb-4f3d-1c3d92aae75c@berlin.de> Inline On 19.05.2017 15:53, forax at univ-mlv.fr wrote: > > > ------------------------------------------------------------------------------------------------------------------------------------ > > *De: *"Stephan Herrmann" > *?: *"John Rose" , jigsaw-dev at openjdk.java.net > *Cc: *"R?mi Forax" > *Envoy?: *Vendredi 19 Mai 2017 12:37:07 > *Objet: *Re: Re: An alternative to "restricted keywords" + helping automatic modules > > A quick question to keep the ball rolling: > > Do we agree on the following assessment of the status quo? > > The definition of "restricted keywords" implies (without explicitly saying so), > that classification of a word as keyword vs. identifier can only be made > *after* parsing has accepted the enclosing ModuleDeclaration. > (With some tweaks, this can be narrowed down to > "after the enclosing ModuleDirective has been accepted") > > This definition is not acceptable. > > > I agree that this is not acceptable but this is not what we are proposing. Who is "we"? Note that your proposal let me conclude that "transitive" is not a legal start of a module reference. If that is not what you intend, please provide a specification-like description of what you have in mind. Probably Stephen's proposal will come in handy for this issue? Your notes about possible implementation may help when we come to implementing, but right now they may also distract from understanding the intention. Stephan > > You do not have to wait the reduction of ModuleDeclaration (or ModuleDirective), the parser know its parsing state (the LR item) > during the parsing not at the end. > The LR analysis is not able to know at some point during the parsing which production will be reduced later but it is able to know > which terminals will not lead to an error when shifting the next terminal. > > When you are in the middle of the parsing, the parser shift a terminal to go from one state to another, so for a state the parser > knows if it can shift by a terminal which is among the set of restricted keywords or not then either it can instruct the lexer > before scanning the token to activate the restricted keyword automata or after having scanned the token it can classify the token as > a keyword instead of as an identifier. > > The idea is that the parser will not only tell when it reduces a production but also when it is about to shift a restricted keyword. > So you can classify a token as an identifier or as a keyword because the parser is able to bubble up that its parser state (the LR > item) may recognize a keyword. > > > > comments? > Stephan > > > R?mi > > > ----- urspr?ngliche Nachricht --------- > > Subject: Re: An alternative to "restricted keywords" + helping automatic modules > Date: Fr 19 Mai 2017 07:27:31 CEST > From: John Rose > To: Stephan Herrmann > > On May 18, 2017, at 1:59 AM, Stephan Herrmann > wrote: > > > In all posts I could not find a real reason against escaping, > aside from aesthetics. I don't see this as sufficient motivation > for a less-then-perfect solution. > > > So, by disregarding esthetics... > > > Clarity: > I'm still not completely following your explanations, partly because > of the jargon you are using. I'll leave it to Alex to decide if he > likes the idea that JLS would have to explain terms like dotted > production. > > Compare this to just adding a few more rules to the grammar, > where no hand-waving is needed for an explanation. > No, I did not say that escaping is a pervasive change. > I never said that the grammar for ordinary compilation units > should be changed. > If you like we only need to extend one rule for the scope of > modular compilation units: Identifier. It can't get simpler. > > > Completeness: > I understand you as saying, module names cannot start with > "transitive". Mind you, that every modifier that will be added > to the grammar for modules in the future will cause conflicts for > names that are now legal, and you won't have a means to resolve this. > > By contrast, we can use the escaping approach even to solve one > more problem that has been briefly touched on this list before: > > Automatic modules suffer from the fact that some artifact names may > have Java keywords in their name, which means that these artifacts > simply cannot be used as automatic modules, right? > Why not apply escaping also here? *Any* dot-separated sequence > of words could be used as module name, as long as module references > have a means to escape any keywords in that sequence. > > > Suitability for implementation: > As said, your proposal resolves one problem, but still IDE > functionality suffers from restricted keywords, because scanning > and parsing need more context information than normal. > > > ?we obtain the freedom for IDEs to disregard abnormal > amounts of context, saving uncounted machine cycles, > > - Recovery after a syntax error will regress. > > > ?and we make life easier for all ten writers of error recovery > functions, > > - Scanning arbitrary regions of code is not possible. > > > ?we unleash the power of an army of grad students to study > bidirectional parsing of module files, > > Remember: > In an IDE code with syntax errors is the norm, not an exception, > as the IDE provides functionality to work on incomplete code. > > > ?and ease the burdens of the thousands who must spend their > time looking at syntax errors for their broken module files. > > Nope, not for me. Give me esthetics, please. Really. > > ? John > > > ---- urspr?ngliche Nachricht Ende ---- > From aguibert at us.ibm.com Fri May 19 15:28:43 2017 From: aguibert at us.ibm.com (Andrew Guibert) Date: Fri, 19 May 2017 10:28:43 -0500 Subject: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/xml/ws/Service In-Reply-To: References: <6d678eef-8eec-448f-9b63-7f2c83083b69@abacus.ch> <0fd04cb7-8d6d-04a6-720a-8b70aa7c99f8@oracle.com> <9ea920d7-f52f-c596-e8f2-349569bc57a1@oracle.com> Message-ID: Hi all, According to JEP 261 the java.se.ee specific modules are being excluded for the sake of app servers since we already typically override them. As a developer of a Java EE app server, while this gesture is appreciated, I think it does more harm than good to the wider Java community because many Java programs that are unrelated to Java EE app servers depended on these APIs in Java 8 and do not override the JDK set with their own versions. Excluding `java.se.ee` specific modules from the root set by default seems to border on violating the "deprecated for at least one release before removal" rule of Java. There was no indication in Java SE 8 [1] that these classes would be removed in Java SE 9. Technically they are not "removed from the JDK"... hey are still there if you add them with the command line arg, but they are not there by default and I think that's the part that matters. Giving framework owners all of JDK 9 as a runway to update their apps to use their own versions of `java.se.ee` modules would be extremely helpful. In my experience in adopting Java 9 and helping others to run on Java 9, the missing "java.se.ee" modules has been the most common "gotcha" of JDK 9 (now that illegal access is permitted by default). Confusingly, the default set of root modules is defined as [2]: - The?java.se?module is a root, if it exists. If it does not exist then every?java.*?module on the upgrade module path or among the system modules that exports at least one package, without qualification, is a root. - Every non-java.*?module on the upgrade module path or among the system modules that exports at least one package, without qualification, is also a root. Which is a long way of saying "everything but java.se.ee modules are a root". Here is a quick list of references I could find to people being confused by the default root module set not including all modules by default: http://stackoverflow.com/questions/43574426/how-to-resolve-java-lang-noclassdeffounderror-javax-xml-bind-jaxbexception-in-j http://stackoverflow.com/questions/42966880/java-lang-noclassdeffounderror-javax-xml-bind-datatypeconverter http://stackoverflow.com/questions/42907605/how-do-i-include-a-java-9-module-at-runtime http://stackoverflow.com/questions/40530898/gradle-does-not-honor-add-modules-jvm-argument-in-jdk9 http://stackoverflow.com/questions/43606502/how-to-add-a-java-9-module-via-manifest-mf http://stackoverflow.com/questions/44051223/which-system-modules-are-on-the-module-path-by-default http://stackoverflow.com/questions/43937401/error-while-running-allure-command-line-generate-command https://bugs.eclipse.org/bugs/show_bug.cgi?id=493761 http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012136.html As a resolution, could we make all observable modules root modules by default? This would greatly simplify the definition of a root module to be: - Every system module or module on the upgrade module path that exports at least one package is a root. In my eyes, the two critical migration issues with JDK 9 are: 1) Encapsulation of JDK apis 2) Not including the "Java EE APIs" in the default set of root modules for the unnamed module (1) has been resolved by making `--illegal-access=permit` default (2) could be resolved with this proposal, or mitigated by having an `Add-Modules` manifest header [1] java.sml.ws.Service javadoc: https://docs.oracle.com/javase/8/docs/api/javax/xml/ws/Service.html [2]: JEP 261 root module definition: http://openjdk.java.net/jeps/261 - Andy > From: Michael Rasmussen > To: Reto Merz > Cc: jigsaw-dev > Date: 05/17/2017 03:13 AM > Subject: Re: AW: Java 9 EA 169 Webstart NoClassDefFoundError: javax/ > xml/ws/Service > Sent by: "jigsaw-dev" > > On 16 May 2017 at 16:52, Reto Merz wrote: > > They need weeks to plan and test an upcoming server-side > > software update before they can install it on the production system. > > Java 9 will be released on 2017/07/27, so we are under high > business pressur. > > Assuming Java 9 follows a similar path at Java 8, it will be around 6 > months after release, before Java 9 is the default download on > java.com, and another 6 months before the auto-updater will update JRE > installations from java 8 to java 9. > Yes, for those users who actively seek out java 9 on day one, there > can be issues, but for the average consumer, there is a 6-12 month > period from release before Java 9 will be on their computers. > > Yes, we still have to go through all the work of migrating to Java 9, > but should the above hold true for Java 9, we do at least have a bit > of breathing room, before it is actively being pushed to users. > > /Michael > From stephan.herrmann at berlin.de Fri May 19 15:38:04 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 19 May 2017 17:38:04 +0200 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: <18317756-ac95-ca17-c614-b6541a426b96@berlin.de> I haven't seen any reaction to this sub-topic: - If an automatic module would have a name containing a Java keyword, is it OK to simply refuse handling this artifact as an automatic module? Stephan On 18.05.2017 10:59, Stephan Herrmann wrote: > Remi, > > I see your proposal as a minimal compromise, avoiding the worst > of difficulties, but I think we can do better. > > Trade-off: > In all posts I could not find a real reason against escaping, > aside from aesthetics. I don't see this as sufficient motivation > for a less-then-perfect solution. > > > Clarity: > I'm still not completely following your explanations, partly because > of the jargon you are using. I'll leave it to Alex to decide if he > likes the idea that JLS would have to explain terms like dotted > production. > > Compare this to just adding a few more rules to the grammar, > where no hand-waving is needed for an explanation. > No, I did not say that escaping is a pervasive change. > I never said that the grammar for ordinary compilation units > should be changed. > If you like we only need to extend one rule for the scope of > modular compilation units: Identifier. It can't get simpler. > > > Completeness: > I understand you as saying, module names cannot start with > "transitive". Mind you, that every modifier that will be added > to the grammar for modules in the future will cause conflicts for > names that are now legal, and you won't have a means to resolve this. > > By contrast, we can use the escaping approach even to solve one > more problem that has been briefly touched on this list before: > > Automatic modules suffer from the fact that some artifact names may > have Java keywords in their name, which means that these artifacts > simply cannot be used as automatic modules, right? > Why not apply escaping also here? *Any* dot-separated sequence > of words could be used as module name, as long as module references > have a means to escape any keywords in that sequence. > > > Suitability for implementation: > As said, your proposal resolves one problem, but still IDE > functionality suffers from restricted keywords, because scanning > and parsing need more context information than normal. > - Recovery after a syntax error will regress. > - Scanning arbitrary regions of code is not possible. > Remember: > In an IDE code with syntax errors is the norm, not an exception, > as the IDE provides functionality to work on incomplete code. > > > Stephan > > > On 18.05.2017 00:34, Remi Forax wrote: >> I want to answer this before we start the meetings because i really think that restricted keyword as i propose solve the issues >> Stephan raised. >> >> >> ----- Mail original ----- >>> De: "Stephan Herrmann" >>> ?: jigsaw-dev at openjdk.java.net >>> Envoy?: Mardi 16 Mai 2017 11:49:45 >>> Objet: Re: An alternative to "restricted keywords" >> >>> Thanks, Remi, for taking this to the EG list. >>> >>> Some collected responses: >>> >>> >>> Remi: "from the user point of view, '^' looks like a hack" >>> >>> This is, of course, a subjective statement. I don't share this view >>> and in years of experience with Xtext-languages (where this concept >>> is used by default) I never heard any user complain about this. >>> >>> More importantly, I hold that such aesthetic considerations are of >>> much lesser significance than the question, whether we can explain >>> - unambiguously explain - the concept in a few simple sentences. >>> Explaining must be possible at two levels: in a rigorous specification >>> and in simple words for users of the language. >> >> I'm not against ^, or ` as it has already asked to escape an identifier, but as you said it's a pervasive change that applies on >> the whole grammar while i think that with restricted keyword (that really should be called local keywords) the changes only impact >> the grammar that specifies a module-info.java >> >>> >>> Remi: "a keyword which is activated if you are at a position in the >>> grammar where it can be recognized". >>> >>> I don't think 'being at a position in the grammar' is a good way of >>> explaining. Parsing doesn't generally have one position in a grammar, >>> multiple productions can be active in the same parser state. >>> Also speaking of a "loop" for modifiers seems to complicate matters >>> more than necessary. >>> >>> Under these considerations I still see '^' as the clearest of all >>> solutions. Clear as a specification, simple to explain to users. >> >> Eclipse uses a LR parser, for a LR parser, position == dotted production as i have written earlier, so no problem because it >> corresponds to only one parser state. Note that even if one do not use an LR or a LL parser, most hand written parser i've seen, >> javac is one of them, also refers to dotted production in the comments of the corresponding methods. >> >>> >>> >>> >>> Peter spoke about module names vs. package names. >>> >>> I think we agree, that module names cannot use "module words", >>> whereas package names should be expected to contain them. >> >> yes, that the main issue, package names may contains unqualified name like 'transitive, ''with' or 'to'. >> but i think people will also want to use existing package or more exactly prefix of existing package as module name, so we should >> also support having restricted keyword name as part of a module name. >> >> The grammar is: >> >> open? module module_name { >> requires (transitive | static)* module_name; >> exports package_name; >> exports package_name to module_name1, module_name2; >> opens package_name; >> opens package_name to module_name1, module_name2; >> uses xxx; >> provides xxx with xxx, yyy; >> } >> >> If we just consider package name, only 'opens' and 'exports' are followed by a package name and a package name can only been >> followed by ';' or 'to', so once 'opens' is parsed, you know that you can have only an identifier so if it's not an identifier by >> one of the restricted keywords, it should be considered as an identifier. >> >> As i said earlier, the scanner can see the restricted keyword as keyword and before feeding the token to the parser, you can check >> the parser state to see if the keyword as to be lowered to an identifier or not. >> >> For module name, there is the supplementary problem of transitive, because if a module starts with transitive, you can have a >> conflict. As i said earlier, instead of using the next token to know if transitive is the keyword or part of the module name, i >> think we should consider it as a keyword, as the JLS said a restricted keyword is activated when it can appear, so "requires >> transitive" is not a valid directive. >> >>> >>> Remi: "you should use reverse DNS naming for package so no problem :)" >>> >>> "to" is a "module word" and a TLD. >>> I think we should be very careful in judging that a existing conflict >>> is not a real problem. Better to clearly and rigorously avoid the >>> conflict in the first place. >> >> to as the first part of a package/module and to as in exports ... to can not be present on the same dotted production, because >> exports as to be followed by a package_name so 'to' here means the start of a package name and then because a package name can not >> ends with '.' you always know if you are inside the production recognizing the package_name or outside matching the to of the >> directive exports. >> >>> >>> >>> >>> Some additional notes from my side: >>> >>> In the escape-approach, it may be prudent to technically allow >>> escaping even words that are identifiers in Java 9, but could become >>> keywords in a future version. This ensures that modules which need >>> more escaping in Java 9+X can still be parsed in Java 9. >> >> yes, that's why i think that escaping is not the right mechanism here, because we want to solve a very local problem so we do not >> need a global grammar-wise way to solve our problem. >> >>> >>> >>> Current focus was on names of modules, packages and types. >>> A complete solution must also give an answer for annotations on modules. >>> Some possible solutions: >>> a. Assume that annotations for modules are designed with modules in mind >>> and thus have to avoid any module words in their names. >>> b. Support escaping also in annotations >>> c. Refine the scope where "module words" are keywords, let it start only >>> when the word "module" or the group "open module" has been consumed. >>> This would make the words "module" and "open" special, as being >>> switch words, where we switch from one language to another. >>> (For this I previously coined the term "scoped keywords" [1]) >> >> For annotation, again, because annotation name are qualified, you now when you see 'module' if you are in the middle of the >> annotation name or if you are outside. >> >>> >>> >>> I think we all agree that the conflicts we are solving here are rare >>> corner cases. Most names do not contain module words. Still, from a >>> conceptual and technical p.o.v. the solution must be bullet proof. >>> But there's no need to be afraid of module declarations being spammed >>> with dozens of '^' characters. Realistically, this will not happen. >>> >> >> I agree, and i strongly believe that scoped keyword, local keywords or restricted keywords, i.e. whatever the name, keywords that >> are keywords or identifiers depending on the parser state are the general mechanism that solve our problem. >> >>> Stephan >>> >>> [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 >>> >> >> R?mi >> > From chris at hazelcast.com Fri May 19 15:42:10 2017 From: chris at hazelcast.com (Christoph Engelbert) Date: Fri, 19 May 2017 17:42:10 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: Message-ID: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> Hey, Just a clarification question, does that mean no agent like dynatrace, appdynamics or others would work anymore with a normal jlinked image, when `java.instrument` is specifically added as a dependency? In this case I would agree with Rafael and Michael. It will come as a surprise to a lot of users / customers which use those APM / monitoring solutions that attach agents into any Java process found. Chris > On 19. May 2017, at 17:07, Michael Rasmussen wrote: > > On 19 May 2017 at 11:22, Alan Bateman wrote: >> One thing that jlink >> could do is emit a warning that the resulting run-time image doesn't have >> the management and instrumentation features, might that be the right >> balance. > > As a users of those kind of agents, and as an agent vendor myself > (though not one I expect to be used often with jlink'ed images - but > how app servers in the future are distributed remains to be seen) I > get where Rafael is coming from. I also agree it will come to a > surprise to many, that if suddenly the distributed image from a > vendor, now created by jlink, no longer included the capability of > attaching agents! > > Having serviceability/agent support included by default makes sense to > me, but I also get the reverse argument, that you should be able to > create a minimalvm/java.base image, if so desired. > > I most definitely think, that jlink should at least emit a warning, if > the image it's generating does not include those features, and also > have the warning include information how to add these features. > A dedicated option for jlink to explicitly enable/disable > serviceability/agent modules could be nice for that. > > /Michael From rafael.wth at gmail.com Fri May 19 15:47:28 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 19 May 2017 17:47:28 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> References: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> Message-ID: Yes, this is an implication and what I am concerned about. Am 19.05.2017 17:42 schrieb "Christoph Engelbert" : Hey, Just a clarification question, does that mean no agent like dynatrace, appdynamics or others would work anymore with a normal jlinked image, when `java.instrument` is specifically added as a dependency? In this case I would agree with Rafael and Michael. It will come as a surprise to a lot of users / customers which use those APM / monitoring solutions that attach agents into any Java process found. Chris > On 19. May 2017, at 17:07, Michael Rasmussen wrote: > > On 19 May 2017 at 11:22, Alan Bateman wrote: >> One thing that jlink >> could do is emit a warning that the resulting run-time image doesn't have >> the management and instrumentation features, might that be the right >> balance. > > As a users of those kind of agents, and as an agent vendor myself > (though not one I expect to be used often with jlink'ed images - but > how app servers in the future are distributed remains to be seen) I > get where Rafael is coming from. I also agree it will come to a > surprise to many, that if suddenly the distributed image from a > vendor, now created by jlink, no longer included the capability of > attaching agents! > > Having serviceability/agent support included by default makes sense to > me, but I also get the reverse argument, that you should be able to > create a minimalvm/java.base image, if so desired. > > I most definitely think, that jlink should at least emit a warning, if > the image it's generating does not include those features, and also > have the warning include information how to add these features. > A dedicated option for jlink to explicitly enable/disable > serviceability/agent modules could be nice for that. > > /Michael From volker.simonis at gmail.com Fri May 19 15:55:10 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 19 May 2017 17:55:10 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: On Fri, May 19, 2017 at 1:05 PM, Alan Bateman wrote: > On 19/05/2017 10:48, Volker Simonis wrote: > >> : >> >> From my understanding, at run-time, "open" implicates "exports" (i.e. >> if a module M1 opens a package P for some other module M2 it also, >> implicitly exports P to M2). The "big kill switch" in both, its old >> and in the newly proposed form, usually only refers to "enabling >> reflective access" but doesn't explicitly mentions that it will also, >> implicitly export the respective packages. Also, the usage of the >> "kill switch" only produces warnings for reflective accesses which are >> enabled by the option itself (and not at the same time, explicitly >> allowed by --add-opens directives). But it doesn't warn about the >> simple, non-reflective accesses to packages which are implicitly >> exported by the kill switch as well. > > You are correct. If the package is opened at run-time: > > 1. All public members of public types in the package are accessible. This > includes both static and reflective access. > > 2. All members of all types in the package can be reflected on when using > setAccessible or privateLookupIn. > > The reason you don't get warnings with the static references is because > access checking in the VM is very low level and we deliberately decided to > avoid the invasive changes needs to report warnings here. The latest > proposal does not change this, it's just too late in JDK 9 to take on this > surgery. > I understand. But than we should at least document it more thoroughly that "--illegal-access=warn" really only warns for reflective accesses and not for static ones (which are implicitly enabled by "--illegal-access=permit"). We must make it clear to developers that running an application warning-free with "--illegal-access=permit/warn" doesn't necessarily mean that the same application will also run correctly with "--illegal-access=deny". Regards, Volker > -Alan From Alan.Bateman at oracle.com Fri May 19 16:02:08 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 May 2017 17:02:08 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> References: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> Message-ID: On 19/05/2017 16:42, Christoph Engelbert wrote: > Hey, > > Just a clarification question, does that mean no agent like dynatrace, appdynamics or others would work anymore with a normal jlinked image, when `java.instrument` is specifically added as a dependency? > > Just to avoid any confusion here: The Oracle JDK downloads include all modules, also the server VM with JVM TI. So there shouldn't be any issues. Same thing for anyone building from the OpenJDK sources. The issue that we discussing here is runtime images created with the `jlink` tool. It's up to the user of `jlink` as to which modules to include and also the VM variants to include. If the user specifies `java.instrument` to jlink then it will be linked in and should just work. -Alan. From anthony.vanelverdinghe at gmail.com Fri May 19 16:36:22 2017 From: anthony.vanelverdinghe at gmail.com (Anthony Vanelverdinghe) Date: Fri, 19 May 2017 18:36:22 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <20170518144840.DE396D342B@eggemoggin.niobe.net> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> Message-ID: <7ef1d4e2-fc1d-2c47-5e41-de1fb6bd4d5a@gmail.com> As I haven't seen it mentioned yet: JDK 9 introduces a new JDK_JAVA_OPTIONS environment variable. So I'd propose to: * have `--illegal-access=deny` be the default * add a `quiet` possibility * not resolve the java.se.ee module by default * add a note to the compatibility guide for JDK 9: if you want maximum compatibility, set `JDK_JAVA_OPTIONS=--illegal-access=quiet --add-modules=java.se.ee`, but be aware that these options will dissapear in a future JDK This way: * there's no need to to add command-line parameters or manifest entries anywhere (and since JDK_JAVA_OPTIONS is new in JDK 9, there's no compatibility impact) * the defaults are already what they ought to be I believe the proposed default is a step backwards, and good arguments have been given against doing this (a.o. by Peter Levart [1]). So the above seems like a good middle ground to me: the defaults are what they should be, and legacy behavior is simply a matter of setting an environment variable. [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012739.html Kind regards, Anthony On 18/05/2017 16:48, mark.reinhold at oracle.com wrote: > Over time, as we've gotten closer and closer to the JDK 9 GA date, more > and more developers have begun paying attention the actual changes in > this release. The strong encapsulation of JDK-internal APIs has, in > particular, triggered many worried expressions of concern that code that > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance > warning of this change was given in JDK 8. > > To help the entire ecosystem migrate to the modular Java platform at a > more relaxed pace I hereby propose to allow illegal reflective access > from code on the class path by default in JDK 9, and to disallow it in > a future release. > > In short, the existing "big kill switch" of the `--permit-illegal-access` > option [1] will become the default behavior of the JDK 9 run-time system, > though without as many warnings. The current behavior of JDK 9, in which > illegal reflective-access operations from code on the class path are not > permitted, will become the default in a future release. Nothing will > change at compile time. > > In detail, the recently-introduced `--permit-illegal-access` option will > be replaced by a more-general option, `--illegal-access`. This option > will take a single keyword parameter, as follows: > > `--illegal-access=permit` > > This will be the default mode for JDK 9. It opens every package in > every explicit module to code in all unnamed modules, i.e., code on > the class path, just as `--permit-illegal-access` does today. > > The first illegal reflective-access operation causes a warning to be > issued, as with `--permit-illegal-access`, but no warnings are issued > after that point. This single warning will describe how to enable > further warnings. > > `--illegal-access=warn` > > This causes a warning message to be issued for each illegal > reflective-access operation. This is equivalent to the current > `--permit-illegal-access` option. > > `--illegal-access=debug` > > This causes both a warning message and a stack trace to be shown > for each illegal reflective-access operation. This is equivalent > to combining today's `--permit-illegal-access` option with > `-Dsun.reflect.debugModuleAccessChecks`. > > `--illegal-access=deny` > > This disables all illegal reflective-access operations except for > those enabled by other command-line options, such as `--add-opens`. > This will become the default mode in a future release. > > Notes: > > - The proposed default mode enables the run-time system to issue a > warning message, possibly at some time long after startup, without > having been explicitly requested to do so. This may be a surprise > in production environments, since it's extremely unusual for the > run-time system to issue any warning messages at all. If the default > mode permits illegal reflective access, however, then it's essential > to make that known so that people aren't surprised when this is no > longer the default mode in a future release. > > - Warning messages in any mode can be avoided, as before, by the > judicious use of the `--add-exports` and `--add-opens` options. > > - This proposal will, if adopted, require adjustments to JEP 260, > "Encapsulate Most Internal APIs" [2]. APIs that are internal to the > JDK will still be strongly encapsulated from the standpoint of code > in modules, whether those modules are automatic or explicit, but they > will not appear to be encapsulated at run time from the standpoint of > code on the class path. > > - When `deny` becomes the default mode then I expect `permit` to remain > supported for at least one release, so that developers can continue > to migrate their code. The `permit`, `warn`, and `debug` modes will, > over time, be removed, as will the `--illegal-access` option itself. > (For launch-script compatibility the unsupported modes will most > likely just be ignored, after issuing a warning to that effect.) > > - This change will not magically solve every JDK 9 adoption problem. > The concrete types of the built-in class loaders are still different, > `rt.jar` is still gone, the layout of a system image is still not the > same, and the version string still has a new format. > > Comments? > > - Mark > > > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html > [2] http://openjdk.java.net/jeps/260 From peter.levart at gmail.com Fri May 19 16:42:00 2017 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 May 2017 18:42:00 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <6ae79561-cbfd-5d2a-a171-66007999ea66@gmail.com> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> <51eb8bfb-19f6-fbd3-bf5b-7effae32611f@oracle.com> <6ae79561-cbfd-5d2a-a171-66007999ea66@gmail.com> Message-ID: Hi, Just a thought... What about keeping the "opens pkg" and "open module m" as is - there is a legitimate argument in keeping all accesses (static and reflective) in sync. But it does not mean that "--illegal-access=permit" must be equivalent to --add-opens all-modules/all-packages=ALL-UNNAMED. It could mean a slightly different thing - just enough to keep the compatibility with JDK 8. There's no point in making jdk.internal.* packages open for example - they are mostly new APIs and not relevant to backwards compatibility. More precisely, let "--illegal-access=permit" allow .setAccessible(true) be called by unnamed module(s) on otherwise inaccessible (private, package-private, protected) members/classes of exported packages only. The reasoning is as follows: - selected JDK modules can explicitly open selected packages if they feel they should keep runtime compatibility with applications but prevent compilation with such APIs (like jdk.unsupported module does for example). In JDK 10 they can close those packages again. - I don't know what percentage of JDK-internal APIs used by applications through reflection is in exported packages and what percent in concealed. It would be interesting to know such statistics first. - User modules can explicitly open selected packages or entire modules if they feel so - User automatic modules already export all packages, but to allow access to their private members via reflection, they should perhaps be equivalent to open modules with all packages exported in addition. What do you think? Regards, Peter On 05/19/2017 04:27 PM, Peter Levart wrote: > > > On 05/19/2017 04:05 PM, Alan Bateman wrote: >> On 19/05/2017 14:54, Peter Levart wrote: >> >>> : >>> >>> Opening the whole JDK (--illegal-access=permit by default) means >>> that all internal "public" APIs are made accessible if by chance >>> someone can grab an instance of target object and/or an instance of >>> Method/Field object. Imagine a JDK developer that thought that by >>> putting a public type into a concealed package was equivalent to >>> making the type module-private. This is a big surprise from the >>> security perspective and jdk.internal.misc.Unsafe.getUnsafe() might >>> not be a lone example. >> True although it's no different to JDK 8 and older behavior where all >> public members of all public types in all packages were accessible to >> code on the class path. > > Except that in the meanwhile a lot of internal code was written for > JDK 9 that assumed the level of privacy provided by concealed > packages. This level is about to be changed by the proposal in the > last minute... > >> Furthermore, setAccessible could be use to hack everywhere. The >> proposal is really just giving libraries and tools more time to sort >> out their issues. >> >> -Alan > > Regards, Peter From forax at univ-mlv.fr Fri May 19 16:51:44 2017 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 19 May 2017 18:51:44 +0200 (CEST) Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: <10567fd5-6776-8beb-4f3d-1c3d92aae75c@berlin.de> References: <1495190227.645288-11404@martha.daybyday.de> <229221312.3399996.1495202027081.JavaMail.zimbra@u-pem.fr> <10567fd5-6776-8beb-4f3d-1c3d92aae75c@berlin.de> Message-ID: <1412932341.3471515.1495212704513.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stephan Herrmann" > ?: forax at univ-mlv.fr, jigsaw-dev at openjdk.java.net > Envoy?: Vendredi 19 Mai 2017 17:26:02 > Objet: Re: An alternative to "restricted keywords" + helping automatic modules > Inline > > On 19.05.2017 15:53, forax at univ-mlv.fr wrote: >> >> >> ------------------------------------------------------------------------------------------------------------------------------------ >> >> *De: *"Stephan Herrmann" >> *?: *"John Rose" , jigsaw-dev at openjdk.java.net >> *Cc: *"R?mi Forax" >> *Envoy?: *Vendredi 19 Mai 2017 12:37:07 >> *Objet: *Re: Re: An alternative to "restricted keywords" + helping automatic >> modules >> >> A quick question to keep the ball rolling: >> >> Do we agree on the following assessment of the status quo? >> >> The definition of "restricted keywords" implies (without explicitly saying so), >> that classification of a word as keyword vs. identifier can only be made >> *after* parsing has accepted the enclosing ModuleDeclaration. >> (With some tweaks, this can be narrowed down to >> "after the enclosing ModuleDirective has been accepted") >> >> This definition is not acceptable. >> >> >> I agree that this is not acceptable but this is not what we are proposing. > > Who is "we"? > > Note that your proposal let me conclude that "transitive" is not a legal > start of a module reference. If that is not what you intend, please provide > a specification-like description of what you have in mind. > Probably Stephen's proposal will come in handy for this issue? transitive is not a valid start of a module name if you want to use it in a requires directive in Java, but it's a valid module name for the JVM, you can create a module-info.class in another language than Java. > > Your notes about possible implementation may help when we come to implementing, > but right now they may also distract from understanding the intention. We have gone into the rabbit hole of talking about implementation because you ask to it's your point (3) ""restricted keywords" pose three problems to tool implementations". The intention is to introduce restricted keywords (i prefer local keywords), to quote the the current draft of the JLS: "They are keywords solely where they appear as terminals in the ModuleDeclaration production (?7.7), and are identifiers everywhere else", so developers will not have to change all their Java codes because open, module, requires, transitive, exports, opens, to, uses, provides, and with are only keywords activated locally in module-info.java. > > Stephan > > R?mi >> >> You do not have to wait the reduction of ModuleDeclaration (or ModuleDirective), >> the parser know its parsing state (the LR item) >> during the parsing not at the end. >> The LR analysis is not able to know at some point during the parsing which >> production will be reduced later but it is able to know >> which terminals will not lead to an error when shifting the next terminal. >> >> When you are in the middle of the parsing, the parser shift a terminal to go >> from one state to another, so for a state the parser >> knows if it can shift by a terminal which is among the set of restricted >> keywords or not then either it can instruct the lexer >> before scanning the token to activate the restricted keyword automata or after >> having scanned the token it can classify the token as >> a keyword instead of as an identifier. >> >> The idea is that the parser will not only tell when it reduces a production but >> also when it is about to shift a restricted keyword. >> So you can classify a token as an identifier or as a keyword because the parser >> is able to bubble up that its parser state (the LR >> item) may recognize a keyword. >> >> >> >> comments? >> Stephan >> >> >> R?mi >> >> >> ----- urspr?ngliche Nachricht --------- >> >> Subject: Re: An alternative to "restricted keywords" + helping automatic modules >> Date: Fr 19 Mai 2017 07:27:31 CEST >> From: John Rose >> To: Stephan Herrmann >> >> On May 18, 2017, at 1:59 AM, Stephan Herrmann > > wrote: >> >> >> In all posts I could not find a real reason against escaping, >> aside from aesthetics. I don't see this as sufficient motivation >> for a less-then-perfect solution. >> >> >> So, by disregarding esthetics... >> >> >> Clarity: >> I'm still not completely following your explanations, partly because >> of the jargon you are using. I'll leave it to Alex to decide if he >> likes the idea that JLS would have to explain terms like dotted >> production. >> >> Compare this to just adding a few more rules to the grammar, >> where no hand-waving is needed for an explanation. >> No, I did not say that escaping is a pervasive change. >> I never said that the grammar for ordinary compilation units >> should be changed. >> If you like we only need to extend one rule for the scope of >> modular compilation units: Identifier. It can't get simpler. >> >> >> Completeness: >> I understand you as saying, module names cannot start with >> "transitive". Mind you, that every modifier that will be added >> to the grammar for modules in the future will cause conflicts for >> names that are now legal, and you won't have a means to resolve this. >> >> By contrast, we can use the escaping approach even to solve one >> more problem that has been briefly touched on this list before: >> >> Automatic modules suffer from the fact that some artifact names may >> have Java keywords in their name, which means that these artifacts >> simply cannot be used as automatic modules, right? >> Why not apply escaping also here? *Any* dot-separated sequence >> of words could be used as module name, as long as module references >> have a means to escape any keywords in that sequence. >> >> >> Suitability for implementation: >> As said, your proposal resolves one problem, but still IDE >> functionality suffers from restricted keywords, because scanning >> and parsing need more context information than normal. >> >> >> ?we obtain the freedom for IDEs to disregard abnormal >> amounts of context, saving uncounted machine cycles, >> >> - Recovery after a syntax error will regress. >> >> >> ?and we make life easier for all ten writers of error recovery >> functions, >> >> - Scanning arbitrary regions of code is not possible. >> >> >> ?we unleash the power of an army of grad students to study >> bidirectional parsing of module files, >> >> Remember: >> In an IDE code with syntax errors is the norm, not an exception, >> as the IDE provides functionality to work on incomplete code. >> >> >> ?and ease the burdens of the thousands who must spend their >> time looking at syntax errors for their broken module files. >> >> Nope, not for me. Give me esthetics, please. Really. >> >> ? John >> >> >> ---- urspr?ngliche Nachricht Ende ---- From stephen.felts at oracle.com Fri May 19 18:30:45 2017 From: stephen.felts at oracle.com (Stephen Felts) Date: Fri, 19 May 2017 11:30:45 -0700 (PDT) Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> Message-ID: Right. I'm asking for a new API so code can be written to avoid the warning. -----Original Message----- From: Alan Bateman Sent: Friday, May 19, 2017 10:11 AM To: Stephen Felts; jigsaw-dev at openjdk.java.net Subject: Re: Proposal: Allow illegal reflective access by default in JDK 9 On 19/05/2017 14:48, Stephen Felts wrote: > By the way, there is no available API so that one can write > > > > If an object is not accessible and setAccessible on the object will > succeed > > call setAccessible > > > > That means that the code needs to call setAccessible and handle the exception, which triggers a warning. > > AccessibleObject canAccess is new in 9 to test if a reflected object is accessible. Another new API is trySetAccessible to avoid using exceptions for control flow. To your example then if setAccessible succeeds in suppressing access to a JDK internal member then you'll get a warning if it's the first illegal access. If setAccessible fails then you'll InaccessibleObjectException will be thrown as it does not, there will be no warning. -Alan. From volker.simonis at gmail.com Fri May 19 18:39:07 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 19 May 2017 20:39:07 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> Message-ID: Hi, without judging any of the proposals in this thread, I just want to clarify some things: - jlink is no standard tool (i.e. it is neither specified nor mandated by the new Java SE 9 specification). This equally applies to the new jimage/jmod file formats. - in contrast, the minimal, compact and full profiles have been specified by the Java SE 8 specification. - the new Java SE 9 specification defines the JPMS together with a set of standard modules and their relation (i.e. dependencies). An implementation of the Java SE 9 specification now only has to implement the "java.base" module. This is a fundamental change compared to previous versions of the specification (excluding maybe only the three profiles in Java SE 8) where Java SE always meant "all of Java SE or nothing". - implementations can further choose to implement an arbitrary "closed subset" of the standard modules defined by Java SE 9. See [1] for a draft of the gory details and the mail threads at [2,3] for recently proposed corrections. - this is all very complicated (technically and from a licensing point of view) because users can not rely on what will be in a Java 9 distribution any more. If they use Oracle/OpenJDK, they can at least use "--list-modules" to get all the observable modules their Java distribution contains. But bear in mind that "--list-modules" is an Oracle/OpenJDK option and not specified and mandated by the standard. - in general, most (if not all) of the command line options are implementation dependent. In the context of command line options "Standard Options" means "Standard Oracle/OpenJDK" options, not options specified by any Java standard. - think of the current problems many people have with the distinction of JDK vs. JRE (e.g. tools,jar, etc). This confusion will multiply with the new Java 9 modular images. I'm afraid, all that won't help you Rafael (and I'm sorry for that!). So why did I wrote all that? I think I just wanted to emphasize the following points: - We need good/better standards (and that's why the JCP is so important)! - With great power comes great responsibility (i.e. as a Java/application provider, use JPMS/jlink wisely)! - As a Java user, insist on getting a "full" JDK (i.e. one which contains the java.se/java.se.ee module)! Regards, Volker [1] http://cr.openjdk.java.net/~iris/se/9/java-se-9-pr-spec-02/java-se-9-spec.html#s7 [2] http://mail.openjdk.java.net/pipermail/java-se-9-spec-experts/2017-May/000020.html [3] http://mail.openjdk.java.net/pipermail/java-se-9-spec-experts/2017-April/thread.html On Fri, May 19, 2017 at 6:02 PM, Alan Bateman wrote: > On 19/05/2017 16:42, Christoph Engelbert wrote: > >> Hey, >> >> Just a clarification question, does that mean no agent like dynatrace, >> appdynamics or others would work anymore with a normal jlinked image, when >> `java.instrument` is specifically added as a dependency? >> >> > Just to avoid any confusion here: The Oracle JDK downloads include all > modules, also the server VM with JVM TI. So there shouldn't be any issues. > Same thing for anyone building from the OpenJDK sources. > > The issue that we discussing here is runtime images created with the `jlink` > tool. It's up to the user of `jlink` as to which modules to include and also > the VM variants to include. If the user specifies `java.instrument` to jlink > then it will be linked in and should just work. > > -Alan. From blackdrag at gmx.org Fri May 19 20:33:56 2017 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 19 May 2017 22:33:56 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <39567732-b32b-6b36-36fc-bca35cc52ab6@codefx.org> <7d267f00-b1b8-9995-4ce6-da46c7c27d8d@gmail.com> <3fb94028-9ef6-614c-fc44-615586560d33@redhat.com> <4769c841-a5b2-463e-85e8-e1d872c2ec7a@default> Message-ID: <591F56B4.4090008@gmx.org> On 19.05.2017 15:38, Stephen Felts wrote: [...] > Here are a few public ones. > > org.python.core.PyJavaClass - Jython > > com.sun.xml.bind.v2.runtime.reflect.opt.Injector ? standalone jaxb > > com.sun.xml.ws.model.Injector ? standalone jaxws > > net.sf.cglib.core.ReflectUtils$ - cglib > > > > Summarizing the use cases: > > 1. The code calls setAccessible and correctly catches and ignores the exception for those that fail. It would be a big change to make this call setAccessible lazily. > > 2. The code continues to use the JDK8 approach and if that fails it falls back to an approach that works on JDK9 and/or other non-Hotspot platforms. In some cases, this code has been around for a decade or more (e.g., to handle platforms without Unsafe). with the same summary you can include Groovy here. bye Jochen From stephan.herrmann at berlin.de Fri May 19 21:35:48 2017 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 19 May 2017 23:35:48 +0200 Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: <1412932341.3471515.1495212704513.JavaMail.zimbra@u-pem.fr> References: <1495190227.645288-11404@martha.daybyday.de> <229221312.3399996.1495202027081.JavaMail.zimbra@u-pem.fr> <10567fd5-6776-8beb-4f3d-1c3d92aae75c@berlin.de> <1412932341.3471515.1495212704513.JavaMail.zimbra@u-pem.fr> Message-ID: <406a0b0f-2172-f5a7-dc70-aaf59d5a8fac@berlin.de> Meanwhile we seem to have (at least) 4 proposals on the table. Here's my biased summary: (A) JLS up until 2017-05-18 PRO: + we have a specification + spec can be interpreted as allowing all module words to be mentionable in all relevant directives. CON: - the specification is interpreted differently by different experts - the interpretation that Alex intends, requires parsing to be done ahead of scanning, which breaks established compiler technology. (B) Remi: in case of ambiguity interpret as keyword, not identifier PRO: + removes the need to parse more than you have scanned. CON: - makes "transitive" unmentionable as the first segment in a module name, and will cause the same effect on any modifier that may be added in the future. - still requires stateful scanning, so typically syntax highlighting will be partly broken, still. (Isn't that an aesthetic aspect, too?) Other IDE functions are affected, too. (C) Remi + Stephen: disambiguate by adding "module" before each module reference. PRO: + removes the need to parse more than you have scanned. + avoids restriction regarding "transitive" and future modifiers. CON: - still requires stateful scanning. Implications see above (D) Stephan: Escape module words to use them as identifier PRO: + all module words can be used in package & module references + allows referring to modules which have Java keywords in their name + avoids adding new technical complexity + clearly specified in the proposal, mostly using standard concepts CON: - some find the occasional escape character (aesthetically) unpleasant People favoring (B) or (C) could further promote their case by providing a (near) formal specification. For anybody interested in further technical implications, I'd be happy to provide pointers to plenty of IDE functions that would be broken (to different degrees) by (A) - (C). When I spoke about bad error recovery, I wasn't complaining about a few more man days of engineering work, but about a conceptual impossibility to achieve the quality that users should expect. Let me add my interpretation of why we are in this strange situation in the first place: The language of module-info is an unusual mix of: - something like a DSL for declaring API and dependencies of modules - a subset of Java If we would take away the complexities of Java annotations and Java comments, nobody would mind hand-coding an arbitrarily tricky parser that easily meets all relevant goals. But nobody will hand-code a parser that is able to parse a significant sub-set of Java. It's the mix of both natures in one language that creates the conflict. Finally, please don't take this as an issue of language design *vs.* tool implementation. We can only make our users happy, if both aspects smoothly integrate, Stephan On 19.05.2017 18:51, forax at univ-mlv.fr wrote: > ----- Mail original ----- >> De: "Stephan Herrmann" >> ?: forax at univ-mlv.fr, jigsaw-dev at openjdk.java.net >> Envoy?: Vendredi 19 Mai 2017 17:26:02 >> Objet: Re: An alternative to "restricted keywords" + helping automatic modules > >> Inline >> >> On 19.05.2017 15:53, forax at univ-mlv.fr wrote: >>> >>> >>> ------------------------------------------------------------------------------------------------------------------------------------ >>> >>> *De: *"Stephan Herrmann" >>> *?: *"John Rose" , jigsaw-dev at openjdk.java.net >>> *Cc: *"R?mi Forax" >>> *Envoy?: *Vendredi 19 Mai 2017 12:37:07 >>> *Objet: *Re: Re: An alternative to "restricted keywords" + helping automatic >>> modules >>> >>> A quick question to keep the ball rolling: >>> >>> Do we agree on the following assessment of the status quo? >>> >>> The definition of "restricted keywords" implies (without explicitly saying so), >>> that classification of a word as keyword vs. identifier can only be made >>> *after* parsing has accepted the enclosing ModuleDeclaration. >>> (With some tweaks, this can be narrowed down to >>> "after the enclosing ModuleDirective has been accepted") >>> >>> This definition is not acceptable. >>> >>> >>> I agree that this is not acceptable but this is not what we are proposing. >> >> Who is "we"? >> >> Note that your proposal let me conclude that "transitive" is not a legal >> start of a module reference. If that is not what you intend, please provide >> a specification-like description of what you have in mind. >> Probably Stephen's proposal will come in handy for this issue? > > transitive is not a valid start of a module name if you want to use it in a requires directive in Java, > but it's a valid module name for the JVM, you can create a module-info.class in another language than Java. > >> >> Your notes about possible implementation may help when we come to implementing, >> but right now they may also distract from understanding the intention. > > We have gone into the rabbit hole of talking about implementation because you ask to it's your point (3) ""restricted keywords" pose three problems to tool implementations". > The intention is to introduce restricted keywords (i prefer local keywords), to quote the the current draft of the JLS: "They are keywords solely where they appear as terminals in the ModuleDeclaration production (?7.7), and are identifiers everywhere else", so developers will not have to change all their Java codes because open, module, requires, transitive, exports, opens, to, uses, provides, and with are only keywords activated locally in module-info.java. > >> >> Stephan >> >> > > R?mi > >>> >>> You do not have to wait the reduction of ModuleDeclaration (or ModuleDirective), >>> the parser know its parsing state (the LR item) >>> during the parsing not at the end. >>> The LR analysis is not able to know at some point during the parsing which >>> production will be reduced later but it is able to know >>> which terminals will not lead to an error when shifting the next terminal. >>> >>> When you are in the middle of the parsing, the parser shift a terminal to go >>> from one state to another, so for a state the parser >>> knows if it can shift by a terminal which is among the set of restricted >>> keywords or not then either it can instruct the lexer >>> before scanning the token to activate the restricted keyword automata or after >>> having scanned the token it can classify the token as >>> a keyword instead of as an identifier. >>> >>> The idea is that the parser will not only tell when it reduces a production but >>> also when it is about to shift a restricted keyword. >>> So you can classify a token as an identifier or as a keyword because the parser >>> is able to bubble up that its parser state (the LR >>> item) may recognize a keyword. >>> >>> >>> >>> comments? >>> Stephan >>> >>> >>> R?mi >>> >>> >>> ----- urspr?ngliche Nachricht --------- >>> >>> Subject: Re: An alternative to "restricted keywords" + helping automatic modules >>> Date: Fr 19 Mai 2017 07:27:31 CEST >>> From: John Rose >>> To: Stephan Herrmann >>> >>> On May 18, 2017, at 1:59 AM, Stephan Herrmann >> > wrote: >>> >>> >>> In all posts I could not find a real reason against escaping, >>> aside from aesthetics. I don't see this as sufficient motivation >>> for a less-then-perfect solution. >>> >>> >>> So, by disregarding esthetics... >>> >>> >>> Clarity: >>> I'm still not completely following your explanations, partly because >>> of the jargon you are using. I'll leave it to Alex to decide if he >>> likes the idea that JLS would have to explain terms like dotted >>> production. >>> >>> Compare this to just adding a few more rules to the grammar, >>> where no hand-waving is needed for an explanation. >>> No, I did not say that escaping is a pervasive change. >>> I never said that the grammar for ordinary compilation units >>> should be changed. >>> If you like we only need to extend one rule for the scope of >>> modular compilation units: Identifier. It can't get simpler. >>> >>> >>> Completeness: >>> I understand you as saying, module names cannot start with >>> "transitive". Mind you, that every modifier that will be added >>> to the grammar for modules in the future will cause conflicts for >>> names that are now legal, and you won't have a means to resolve this. >>> >>> By contrast, we can use the escaping approach even to solve one >>> more problem that has been briefly touched on this list before: >>> >>> Automatic modules suffer from the fact that some artifact names may >>> have Java keywords in their name, which means that these artifacts >>> simply cannot be used as automatic modules, right? >>> Why not apply escaping also here? *Any* dot-separated sequence >>> of words could be used as module name, as long as module references >>> have a means to escape any keywords in that sequence. >>> >>> >>> Suitability for implementation: >>> As said, your proposal resolves one problem, but still IDE >>> functionality suffers from restricted keywords, because scanning >>> and parsing need more context information than normal. >>> >>> >>> ?we obtain the freedom for IDEs to disregard abnormal >>> amounts of context, saving uncounted machine cycles, >>> >>> - Recovery after a syntax error will regress. >>> >>> >>> ?and we make life easier for all ten writers of error recovery >>> functions, >>> >>> - Scanning arbitrary regions of code is not possible. >>> >>> >>> ?we unleash the power of an army of grad students to study >>> bidirectional parsing of module files, >>> >>> Remember: >>> In an IDE code with syntax errors is the norm, not an exception, >>> as the IDE provides functionality to work on incomplete code. >>> >>> >>> ?and ease the burdens of the thousands who must spend their >>> time looking at syntax errors for their broken module files. >>> >>> Nope, not for me. Give me esthetics, please. Really. >>> >>> ? John >>> >>> >>> ---- urspr?ngliche Nachricht Ende ---- From forax at univ-mlv.fr Sat May 20 12:59:18 2017 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 20 May 2017 14:59:18 +0200 (CEST) Subject: An alternative to "restricted keywords" + helping automatic modules In-Reply-To: References: <2bdf5cbb-b3a1-dc16-702f-af7fc9b4c645@berlin.de> <641041596.2374187.1494569855008.JavaMail.zimbra@u-pem.fr> <7d3d6f28-5560-6e00-81fc-b7562a3d148d@gmail.com> <6ED388E7-1274-4813-9FE9-A2757F6A409E@univ-mlv.fr> <8bc6cebc-23e1-0ddd-9939-3ce13b9fd4c5@berlin.de> <1232913235.2073916.1495060481590.JavaMail.zimbra@u-pem.fr> Message-ID: <1529108085.3727081.1495285158589.JavaMail.zimbra@u-pem.fr> Hi Stephen, ----- Mail original ----- > De: "Stephen Colebourne" > ?: "jigsaw-dev" > Envoy?: Vendredi 19 Mai 2017 10:58:01 > Objet: Re: An alternative to "restricted keywords" + helping automatic modules > I don't support the ^ element or escaping like that either. > > However, would adding a "module" keyword help? > > module com.foo.lib { > requires module com.foo.bar; > } > > thus: > > module com.foo.lib { > requires static module blah; > requires transitive module transitive; > } > > ie. the module name is always prefixed by "module" in a "requires" > statement. But does this help? I will only answer on this question from the technical point of view, the EG as already rule out this syntax and i do not think it's wise to reopen an issue on the module-info syntax now. So yes it helps but we don't care. Adding any mandatory keyword between the modifiers and the module name in the require directive as the effect to allow to require a module with module name that starts with transitive. But this is solve a corner case of a corner case. The corner case we are trying to solve here is to allow package name and module name that contains any keywords defined in module-info that are not defined as keywords in JLS8, (open, module, requires, transitive, exports, opens, to, uses, provides, and with). For that we need, a way to activate keywords locally when the grammar wait for them and do not activate them where we don't care (in the middle of a package name or a module name). The fact that activating keywords do not solve the problem of module that starts with transitive in the require directive should be seen in my point of view a corner case. And as i already said, because module name should use reverse DNS convention, i'm not even sure this problem will occur in practice, thus here we can just say that we will not solve that issue. > > Stephen > > regards, R?mi > On 18 May 2017 at 09:59, Stephan Herrmann wrote: >> Remi, >> >> I see your proposal as a minimal compromise, avoiding the worst >> of difficulties, but I think we can do better. >> >> Trade-off: >> In all posts I could not find a real reason against escaping, >> aside from aesthetics. I don't see this as sufficient motivation >> for a less-then-perfect solution. >> >> >> Clarity: >> I'm still not completely following your explanations, partly because >> of the jargon you are using. I'll leave it to Alex to decide if he >> likes the idea that JLS would have to explain terms like dotted >> production. >> >> Compare this to just adding a few more rules to the grammar, >> where no hand-waving is needed for an explanation. >> No, I did not say that escaping is a pervasive change. >> I never said that the grammar for ordinary compilation units >> should be changed. >> If you like we only need to extend one rule for the scope of >> modular compilation units: Identifier. It can't get simpler. >> >> >> Completeness: >> I understand you as saying, module names cannot start with >> "transitive". Mind you, that every modifier that will be added >> to the grammar for modules in the future will cause conflicts for >> names that are now legal, and you won't have a means to resolve this. >> >> By contrast, we can use the escaping approach even to solve one >> more problem that has been briefly touched on this list before: >> >> Automatic modules suffer from the fact that some artifact names may >> have Java keywords in their name, which means that these artifacts >> simply cannot be used as automatic modules, right? >> Why not apply escaping also here? *Any* dot-separated sequence >> of words could be used as module name, as long as module references >> have a means to escape any keywords in that sequence. >> >> >> Suitability for implementation: >> As said, your proposal resolves one problem, but still IDE >> functionality suffers from restricted keywords, because scanning >> and parsing need more context information than normal. >> - Recovery after a syntax error will regress. >> - Scanning arbitrary regions of code is not possible. >> Remember: >> In an IDE code with syntax errors is the norm, not an exception, >> as the IDE provides functionality to work on incomplete code. >> >> >> Stephan >> >> >> On 18.05.2017 00:34, Remi Forax wrote: >>> >>> I want to answer this before we start the meetings because i really think >>> that restricted keyword as i propose solve the issues Stephan raised. >>> >>> >>> ----- Mail original ----- >>>> >>>> De: "Stephan Herrmann" >>>> ?: jigsaw-dev at openjdk.java.net >>>> Envoy?: Mardi 16 Mai 2017 11:49:45 >>>> Objet: Re: An alternative to "restricted keywords" >>> >>> >>>> Thanks, Remi, for taking this to the EG list. >>>> >>>> Some collected responses: >>>> >>>> >>>> Remi: "from the user point of view, '^' looks like a hack" >>>> >>>> This is, of course, a subjective statement. I don't share this view >>>> and in years of experience with Xtext-languages (where this concept >>>> is used by default) I never heard any user complain about this. >>>> >>>> More importantly, I hold that such aesthetic considerations are of >>>> much lesser significance than the question, whether we can explain >>>> - unambiguously explain - the concept in a few simple sentences. >>>> Explaining must be possible at two levels: in a rigorous specification >>>> and in simple words for users of the language. >>> >>> >>> I'm not against ^, or ` as it has already asked to escape an identifier, >>> but as you said it's a pervasive change that applies on the whole grammar >>> while i think that with restricted keyword (that really should be called >>> local keywords) the changes only impact the grammar that specifies a >>> module-info.java >>> >>>> >>>> Remi: "a keyword which is activated if you are at a position in the >>>> grammar where it can be recognized". >>>> >>>> I don't think 'being at a position in the grammar' is a good way of >>>> explaining. Parsing doesn't generally have one position in a grammar, >>>> multiple productions can be active in the same parser state. >>>> Also speaking of a "loop" for modifiers seems to complicate matters >>>> more than necessary. >>>> >>>> Under these considerations I still see '^' as the clearest of all >>>> solutions. Clear as a specification, simple to explain to users. >>> >>> >>> Eclipse uses a LR parser, for a LR parser, position == dotted production >>> as i have written earlier, so no problem because it corresponds to only one >>> parser state. Note that even if one do not use an LR or a LL parser, most >>> hand written parser i've seen, javac is one of them, also refers to dotted >>> production in the comments of the corresponding methods. >>> >>>> >>>> >>>> >>>> Peter spoke about module names vs. package names. >>>> >>>> I think we agree, that module names cannot use "module words", >>>> whereas package names should be expected to contain them. >>> >>> >>> yes, that the main issue, package names may contains unqualified name like >>> 'transitive, ''with' or 'to'. >>> but i think people will also want to use existing package or more exactly >>> prefix of existing package as module name, so we should also support having >>> restricted keyword name as part of a module name. >>> >>> The grammar is: >>> >>> open? module module_name { >>> requires (transitive | static)* module_name; >>> exports package_name; >>> exports package_name to module_name1, module_name2; >>> opens package_name; >>> opens package_name to module_name1, module_name2; >>> uses xxx; >>> provides xxx with xxx, yyy; >>> } >>> >>> If we just consider package name, only 'opens' and 'exports' are followed >>> by a package name and a package name can only been followed by ';' or 'to', >>> so once 'opens' is parsed, you know that you can have only an identifier so >>> if it's not an identifier by one of the restricted keywords, it should be >>> considered as an identifier. >>> >>> As i said earlier, the scanner can see the restricted keyword as keyword >>> and before feeding the token to the parser, you can check the parser state >>> to see if the keyword as to be lowered to an identifier or not. >>> >>> For module name, there is the supplementary problem of transitive, because >>> if a module starts with transitive, you can have a conflict. As i said >>> earlier, instead of using the next token to know if transitive is the >>> keyword or part of the module name, i think we should consider it as a >>> keyword, as the JLS said a restricted keyword is activated when it can >>> appear, so "requires transitive" is not a valid directive. >>> >>>> >>>> Remi: "you should use reverse DNS naming for package so no problem :)" >>>> >>>> "to" is a "module word" and a TLD. >>>> I think we should be very careful in judging that a existing conflict >>>> is not a real problem. Better to clearly and rigorously avoid the >>>> conflict in the first place. >>> >>> >>> to as the first part of a package/module and to as in exports ... to can >>> not be present on the same dotted production, because exports as to be >>> followed by a package_name so 'to' here means the start of a package name >>> and then because a package name can not ends with '.' you always know if you >>> are inside the production recognizing the package_name or outside matching >>> the to of the directive exports. >>> >>>> >>>> >>>> >>>> Some additional notes from my side: >>>> >>>> In the escape-approach, it may be prudent to technically allow >>>> escaping even words that are identifiers in Java 9, but could become >>>> keywords in a future version. This ensures that modules which need >>>> more escaping in Java 9+X can still be parsed in Java 9. >>> >>> >>> yes, that's why i think that escaping is not the right mechanism here, >>> because we want to solve a very local problem so we do not need a global >>> grammar-wise way to solve our problem. >>> >>>> >>>> >>>> Current focus was on names of modules, packages and types. >>>> A complete solution must also give an answer for annotations on modules. >>>> Some possible solutions: >>>> a. Assume that annotations for modules are designed with modules in mind >>>> and thus have to avoid any module words in their names. >>>> b. Support escaping also in annotations >>>> c. Refine the scope where "module words" are keywords, let it start only >>>> when the word "module" or the group "open module" has been consumed. >>>> This would make the words "module" and "open" special, as being >>>> switch words, where we switch from one language to another. >>>> (For this I previously coined the term "scoped keywords" [1]) >>> >>> >>> For annotation, again, because annotation name are qualified, you now when >>> you see 'module' if you are in the middle of the annotation name or if you >>> are outside. >>> >>>> >>>> >>>> I think we all agree that the conflicts we are solving here are rare >>>> corner cases. Most names do not contain module words. Still, from a >>>> conceptual and technical p.o.v. the solution must be bullet proof. >>>> But there's no need to be afraid of module declarations being spammed >>>> with dozens of '^' characters. Realistically, this will not happen. >>>> >>> >>> I agree, and i strongly believe that scoped keyword, local keywords or >>> restricted keywords, i.e. whatever the name, keywords that are keywords or >>> identifiers depending on the parser state are the general mechanism that >>> solve our problem. >>> >>>> Stephan >>>> >>>> [1] http://www.objectteams.org/def/1.3/sA.html#sA.0.1 >>>> >>> >>> R?mi >>> From Alan.Bateman at oracle.com Sun May 21 09:20:33 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 21 May 2017 10:20:33 +0100 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: References: <20170518144840.DE396D342B@eggemoggin.niobe.net> <4bdbb8af-c597-47ef-307d-5e80b3cee0d6@gmail.com> <56a083e6-3752-ba2c-694c-e22085bd7c12@oracle.com> <51eb8bfb-19f6-fbd3-bf5b-7effae32611f@oracle.com> <6ae79561-cbfd-5d2a-a171-66007999ea66@gmail.com> Message-ID: On 19/05/2017 17:42, Peter Levart wrote: > : > > But it does not mean that "--illegal-access=permit" must be equivalent > to --add-opens all-modules/all-packages=ALL-UNNAMED. It could mean a > slightly different thing - just enough to keep the compatibility with > JDK 8. There's no point in making jdk.internal.* packages open for > example - they are mostly new APIs and not relevant to backwards > compatibility. This is a good point as it would of course be a big set back to open new packages (such as jdk.internal.misc as we have been discussing here). The JDK 9 build has the list of the JDK 8 internal packages (as it is needed for other tools) so it is feasible to not export the internal packages that are "new" in JDK 9. We need to give the proposal a few days to collect feedback before bringing up all the detailed issues around this. > : > > - selected JDK modules can explicitly open selected packages if they > feel they should keep runtime compatibility with applications but > prevent compilation with such APIs (like jdk.unsupported module does > for example). In JDK 10 they can close those packages again. The JDK modules aren't going to export these packages so anyone compiling code with JDK 9 and static references to types in these packages will need to compile with `--add-exports`. You are right that a possible option is to open specific packages but that has lots of implications, particularly for the standard modules. It would also be problematic for `--illegal-access=deny`, something that is important to have in 9 to test how code will behave when the JDK internals are eventually encapsulated. > - I don't know what percentage of JDK-internal APIs used by > applications through reflection is in exported packages and what > percent in concealed. It would be interesting to know such statistics > first. We've presented data on static references to sun.* APIs at conferences as it's not hard to do static analysis to get this data. It's much harder to get data on code using reflection to get at JDK internals. Early builds allowed deep reflection of all members of all types in exported packages - that was the initial proposal in 2015 to keep as much existing bad code working as possible. Fixing that issue (#AwkwardStrongEncapsulation) exposed a lot more hacks with code that is breaking into protected and private members of public types in exported packages. There was a notable increase in the hate mail with that change. There is also the anti-pattern that is obj.getClass().invoke(...). I've tried of dozens of libraries and frameworks to identify the hacks. In general they are all over the map but there is a concentration of hacks on protected and private members of java.base's exported packages. > - User modules can explicitly open selected packages or entire modules > if they feel so Yes. > - User automatic modules already export all packages, but to allow > access to their private members via reflection, they should perhaps be > equivalent to open modules with all packages exported in addition. Automatic modules have always opened all packages so I don't think there is an issue there. -Alan From reto.merz at abacus.ch Sun May 21 20:46:09 2017 From: reto.merz at abacus.ch (Reto Merz) Date: Sun, 21 May 2017 22:46:09 +0200 Subject: Proposal: Allow illegal reflective access by default in JDK 9 In-Reply-To: Message-ID: <750748271-23983@kerio.abacus.ch> On 05/19/2017 11:48 AM, Volker Simonis wrote: > Hi Mark, > > thanks for this proposal. I think it is pragmatic and well balanced. > > While I personally agree with the advocates of a clean solution I also > understand the the needs of big (and slow) companies (including the > one I'm working for:) After all, the new default isn't that bad in my > opinion because it only affects applications/libraries on the class > path which is legacy anyway. If you convert your application/libraries > to modules it will still benefit from the full JPMS features and > restrictions. So altogether I think it is a good compromise. I agree with Volker. Thanks Reto From javadevlist at javasoft.de Mon May 22 07:45:34 2017 From: javadevlist at javasoft.de (wzberger) Date: Mon, 22 May 2017 09:45:34 +0200 Subject: How to load resources from base module In-Reply-To: <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> Message-ID: Hm, so how can ClassA access the resource from module com.b directly - without having a class instance of module b? -Wolfgang > On 18/05/2017 08:16, wzberger wrote: > >> : >> >> public class ClassA{ >> public ClassA() { >> //success >> System.out.println(getClass().getResource("/com/b/resources/test.txt")); > The test creates an instance of ClassB (extends ClassA) and so > getClass() returns ClassB here, not ClassA. So when constructing a > ClassB then this is equivalent to ClassB.class.getResource(...). > >> >> //fail - null >> System.out.println(ClassA.class.getResource("/com/b/resources/test.txt")); >> > Right, as this resource is not in the com.a module. Change this to > ClassB.class.getResource(...) and it should work. > > -Alan > > From Alan.Bateman at oracle.com Mon May 22 07:55:03 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 May 2017 08:55:03 +0100 Subject: How to load resources from base module In-Reply-To: References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> Message-ID: <399befa5-923f-b008-cf1f-c0197159758f@oracle.com> On 22/05/2017 08:45, wzberger wrote: > Hm, so how can ClassA access the resource from module com.b directly - > without having a class instance of module b? Does the code in ClassA have any context to work this? How does it know the resource is in com.b? The APIs allow you to locate a resource in com.b if you have a reference to a class in com.b or the Module object for com.b. -Alan From javadevlist at javasoft.de Mon May 22 08:05:35 2017 From: javadevlist at javasoft.de (wzberger) Date: Mon, 22 May 2017 10:05:35 +0200 Subject: How to load resources from base module In-Reply-To: <399befa5-923f-b008-cf1f-c0197159758f@oracle.com> References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> <399befa5-923f-b008-cf1f-c0197159758f@oracle.com> Message-ID: ClassA acts as a kind of resource loader and only the resource string is passed - is it possible to include the module name in the resource string something like "module!resource"? -Wolfgang Am 22.05.2017 um 09:55 schrieb Alan Bateman: > On 22/05/2017 08:45, wzberger wrote: > >> Hm, so how can ClassA access the resource from module com.b directly >> - without having a class instance of module b? > Does the code in ClassA have any context to work this? How does it > know the resource is in com.b? The APIs allow you to locate a resource > in com.b if you have a reference to a class in com.b or the Module > object for com.b. > > -Alan > > From Alan.Bateman at oracle.com Mon May 22 08:30:27 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 May 2017 09:30:27 +0100 Subject: How to load resources from base module In-Reply-To: References: <93f2062f-19b4-c8b9-f088-6de03291a013@javasoft.de> <9ed8c06c-0b34-9958-245e-271b75867b17@javasoft.de> <6023afd5-dfbc-694c-d658-9d8a11180ed1@oracle.com> <6f629a5f-a429-a997-b134-3d111a1196e8@oracle.com> <399befa5-923f-b008-cf1f-c0197159758f@oracle.com> Message-ID: <24b566dd-fb37-2981-2428-fdc00b0c9761@oracle.com> On 22/05/2017 09:05, wzberger wrote: > ClassA acts as a kind of resource loader and only the resource string > is passed - is it possible to include the module name in the resource > string something like "module!resource"? The module name isn't unique either. Can you say a bit more about ClassA? It is really just a wrapper around Class.getResource or does it do something with the resource. Could the classA API take Supplier or Supplier so that the resource lookup is actually executed in the context of classA's caller? Alternatively, if the code ClassA insists on calling Class.getResource then would it be a burden to have the callers pass in some context - in the example then code in com.b could call this API with a type token or the Module object for com.b. -Alan From adinn at redhat.com Mon May 22 10:49:49 2017 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 22 May 2017 11:49:49 +0100 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: References: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> Message-ID: <8e57c0d0-3d37-0ec8-1444-b3fce5aa7209@redhat.com> Hi Volker, On 19/05/17 19:39, Volker Simonis wrote: > So why did I wrote all that? I think I just wanted to emphasize the > following points: > > - We need good/better standards (and that's why the JCP is so important)! > - With great power comes great responsibility (i.e. as a > Java/application provider, use JPMS/jlink wisely)! > - As a Java user, insist on getting a "full" JDK (i.e. one which > contains the java.se/java.se.ee module)! I agree with all these recommendations. However, I want to just comment on the second one, specifically wrt jlink. What constitutes a /wise/ use of jlink really depends on the purpose of the jlinked image. I can envisage many cases where a custom jlinked image without instrumentation capabilities might look attractive as a way to save on memory footprint (not just the java classes but also the code needed in libjvm.so). Footprint is a significant issue for certain classes of application. The sort of case I envision where this concern would be legitimate would be, say, a shrink-wrapped Java utility program that does a very specific and, probably, quite short-lived job; one that has been well tested and validated, first in a sandbox and then in the field. A more general purpose image, in particular one that is likely to run an app that installs runtime defined deployments that are not linked in the base image -- even if that is only a small amount of code -- may well run up against problems that only an agent can help diagnose and remedy. So, I can't see it being a wise idea to omit agent support in such images. Lastly, I'll add that I don't believe agent developers have any right to /dictate/ to users of jlink what they should or should not do. The onus is for us to make them aware of what they might be missing by ruling out use of our tools and make those tools so compelling that they recognise and agree that the benefits are worth preserving. As long as the option to include or exclude agent support is available, then to me it's simply a free and open market, trading quality and reliability at the potential cost of footprint. I'm very happy to compete on those terms. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From alan.bateman at oracle.com Mon May 22 11:19:04 2017 From: alan.bateman at oracle.com (alan.bateman at oracle.com) Date: Mon, 22 May 2017 11:19:04 +0000 Subject: hg: jigsaw/jake/jdk: Javadoc typos Message-ID: <201705221119.v4MBJ4qd007189@aojmv0008.oracle.com> Changeset: 44dc458dd4ba Author: alanb Date: 2017-05-22 12:16 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/44dc458dd4ba Javadoc typos ! src/java.base/share/classes/java/lang/module/ModuleFinder.java ! src/java.instrument/share/classes/java/lang/instrument/package.html From rafael.wth at gmail.com Mon May 22 21:23:51 2017 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 22 May 2017 23:23:51 +0200 Subject: Attaching to a JVM image that does not include java.instrument In-Reply-To: <8e57c0d0-3d37-0ec8-1444-b3fce5aa7209@redhat.com> References: <7EE5DA44-700D-4E7D-8AF5-E14FD2492F1D@hazelcast.com> <8e57c0d0-3d37-0ec8-1444-b3fce5aa7209@redhat.com> Message-ID: I do not think that people would normally try to dictate it but there is a good chance that people follow the default setting and ignore the warning message. I rarely see a build without warnings, people normally look away. The problem I see is that the people in charge of building and packaging the application do not necessarily understand the purpose of JVMTI with regards to tooling. If such an image is distributed, this has an impact on the users of the image which often require agents to monitor their environments. Basically any customer of mine that I have worked with in the last 5 years is using such agent-based monitoring. To overcome this limitation, a production team could of course ask the team that is maintaining the image to add the module and release an updated version, but by my experience, such coordination is difficult and expensive no matter how trivial it sounds. It is a great feature of such tooling to "just work" which is now mitigated. Personally, I fear that this could introduce some form of cluttering in the Java market. Few people understand JVMTI or even know about it. To those people, it would just seem like this tooling only sometimes works with Java processes, making the platform appear less reliable. And for maintainers of such tooling, there is no way around other than patching the image prior to attachment (what is possible by copying some files around, but it feels like a hack). JVMTI has become an essential feature of the JVM for a reason. I still hope it would be a part of any jlink