From bear.amade at gmail.com Fri Mar 2 09:22:18 2018 From: bear.amade at gmail.com (Bernard Amade) Date: Fri, 2 Mar 2018 10:22:18 +0100 Subject: suggested document enhancement Message-ID: Hello all since my job is to write java tutorials I have to confess I am lost on some topics about the module system. Here is a first concern: - I used to teach to deploy resources in an additional jar at deploy time - so if a code requesting a resource was in package/directory com.mycomp.biz then the resource was in a com.mycomp.biz.config directory ... in a different jar which could be written by the "deployment programmer". with the module system I do not know how to have something similar. Well may be the resource could be in a non-modular jar in the class-path (or added by patch-module) but realy I don't know what could be the best option (put it in a modular jar? but then how to write the module-info, how to update the module-info of other modules - something done by the deployment programmer- ? how to start the jar: from the command line? but then can we still have a "clickable" jar? the doc at http://openjdk.java.net/projects/jigsaw/quick-start is good but lacks this kind of detail. Thanks for any hint From Alan.Bateman at oracle.com Fri Mar 2 10:59:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 2 Mar 2018 10:59:14 +0000 Subject: suggested document enhancement In-Reply-To: References: Message-ID: <51d46e16-c307-38ed-4557-51bc92a37ae4@oracle.com> On 02/03/2018 09:22, Bernard Amade wrote: > Hello all > since my job is to write java tutorials I have to confess I am lost on some topics about the module system. > Here is a first concern: > - I used to teach to deploy resources in an additional jar at deploy time > - so if a code requesting a resource was in package/directory com.mycomp.biz then the resource was in a com.mycomp.biz.config directory ... in a different jar which could be written by the "deployment programmer". > > with the module system I do not know how to have something similar. > Well may be the resource could be in a non-modular jar in the class-path (or added by patch-module) > but realy I don't know what could be the best option (put it in a modular jar? but then how to write the module-info, how to update the module-info of other modules - something done by the deployment programmer- ? how to start the jar: from the command line? but then can we still have a "clickable" jar? > Resources aren't a first class concept in the module system so we of course recommend using services itself. If you are migrating existing code without doing any significant rearchitecture then modules can open packages to allow resources be located by code in other modules with the legacy ClassLoader.getResourceXXX APIs. There is also support in the ResourceBundle API to use resource bundles in other modules. The Quick Start Guide is indeed a bit minimal. To learn more then the presentations and recording from several conferences are linked from the Project Jigsaw page and may be useful to you. In addition, there are a number of books on the topic. In particular, "Java 9 Modularity", by Sander Mak and Paul Bakker, is an excellent resource that will stand the test of time. That book has a section entitled "Resource Encapsulation" that should help you get going. I assume by "clickable jar" you mean executable JARs, as in `java -jar app.jar`. There is no support at this time for starting modular applications with the `java -jar`. -Alan From bear.amade at gmail.com Fri Mar 2 14:14:38 2018 From: bear.amade at gmail.com (Bernard Amade) Date: Fri, 2 Mar 2018 15:14:38 +0100 Subject: modules and resources [ was: suggested document enhancement] In-Reply-To: <51d46e16-c307-38ed-4557-51bc92a37ae4@oracle.com> References: <51d46e16-c307-38ed-4557-51bc92a37ae4@oracle.com> Message-ID: <12B8E301-A1C1-411E-87AF-61830EC95A8C@gmail.com> thanks for replying > Le 2 mars 2018 ? 11:59, Alan Bateman a ?crit : >> > Resources aren't a first class concept in the module system so we of course recommend using services itself. so you mean reconsider the resource handling as a spi (as it is done with ResourceBundle) explaining things to students will get pretty complicated since it should be split between resources located in the module and resources from external modules BTW: another option may be that the "deployment programmer" updates the modular jar ( some signature problems?) or invoke with "patch-module" but is this politically correct ? ok but why is the "old" way deprecated ? ( "first class" concept?) > > If you are migrating existing code without doing any significant rearchitecture then modules can open packages to allow resources be located by code in other modules with the legacy ClassLoader.getResourceXXX APIs. There is also support in the ResourceBundle API to use resource bundles in other modules. ClassLoader.getResource still works with resources located in other modules? the API doc says ... "this method will only find resources in packages of named modules when the package is opened unconditionally" for instance if I have a package with no code but resources only then IntelliJ does not want me to describe it in the module-info ... is this a spec or an IDE quirk? > > . In particular, "Java 9 Modularity", by Sander Mak and Paul Bakker, is an excellent resource that will stand the test of time. That book has a section entitled "Resource Encapsulation" that should help you get going. ok bought the book right now (but it will be shipped only in a fortnight ;-( ) > > I assume by "clickable jar" you mean executable JARs, as in `java -jar app.jar`. There is no support at this time for starting modular applications with the `java -jar`. Hmm. 1) you should explicitly document that so poor guys like me will not spend hours trying to understand what is happening (java's man does not states that!) 2) then what is the simplest way to deploy jars if clicking on a jar does not work anymore with modules? (scripts are platform dependent, so are Jlink images ,...and what happens with jars delivered through java webstart? should we consider that java webstart is going to be deprecated?) thanks - Bear From blackdrag at gmx.org Fri Mar 2 14:43:15 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 2 Mar 2018 15:43:15 +0100 Subject: modules and resources [ was: suggested document enhancement] In-Reply-To: <12B8E301-A1C1-411E-87AF-61830EC95A8C@gmail.com> References: <51d46e16-c307-38ed-4557-51bc92a37ae4@oracle.com> <12B8E301-A1C1-411E-87AF-61830EC95A8C@gmail.com> Message-ID: <5fcafdd4-af81-799f-6812-29ab8d89bbf5@gmx.org> Am 02.03.2018 um 15:14 schrieb Bernard Amade: [...] > 2) then what is the simplest way to deploy jars if clicking on a jar does not work anymore with modules? > (scripts are platform dependent, so are Jlink images ,...and what happens with jars delivered through java webstart? should we consider that java webstart is going to be deprecated?) from http://www.oracle.com/technetwork/java/javase/9-deprecated-features-3745636.html > docs/release_notes > Java Deployment Technologies are deprecated and will be removed in a future release > Java Applet and WebStart functionality, including the Applet API, The Java plug-in, the Java Applet Viewer, JNLP and Java Web Start including the javaws tool are all deprecated in JDK 9 and will be removed in a future release So they are deprecated in 9 according to this, I think 10 prints a warning still. bye Jochen From bear.amade at gmail.com Sat Mar 3 09:49:05 2018 From: bear.amade at gmail.com (Bernard Amade) Date: Sat, 3 Mar 2018 10:49:05 +0100 Subject: modules and resources [ was: suggested document enhancement] In-Reply-To: <5fcafdd4-af81-799f-6812-29ab8d89bbf5@gmx.org> References: <51d46e16-c307-38ed-4557-51bc92a37ae4@oracle.com> <12B8E301-A1C1-411E-87AF-61830EC95A8C@gmail.com> <5fcafdd4-af81-799f-6812-29ab8d89bbf5@gmx.org> Message-ID: <2BC97C47-38D7-4653-B9CE-F2212CE05CBC@gmail.com> I suspected that webstart will be deprecated but ... that raises interesting questions: - why is the java 9 doc not up to date? https://docs.oracle.com/javase/9/deploy/toc.html (explicitly describes webstart) - what would be a (future?) technology to update deliveries? (needed!) (and btw this: https://docs.oracle.com/javase/9/docs/specs/jar/jar.html is not also up to date since it does not says anywhere that -jar option was not compatible with modular jars ) thanks > Le 2 mars 2018 ? 15:43, Jochen Theodorou a ?crit : > > > > Am 02.03.2018 um 15:14 schrieb Bernard Amade: > [...] >> 2) then what is the simplest way to deploy jars if clicking on a jar does not work anymore with modules? >> (scripts are platform dependent, so are Jlink images ,...and what happens with jars delivered through java webstart? should we consider that java webstart is going to be deprecated?) > > from http://www.oracle.com/technetwork/java/javase/9-deprecated-features-3745636.html > >> docs/release_notes >> Java Deployment Technologies are deprecated and will be removed in a future release Java Applet and WebStart functionality, including the Applet API, The Java plug-in, the Java Applet Viewer, JNLP and Java Web Start including the javaws tool are all deprecated in JDK 9 and will be removed in a future release > > So they are deprecated in 9 according to this, I think 10 prints a warning still. > > bye Jochen > From bear.amade at gmail.com Mon Mar 5 09:54:41 2018 From: bear.amade at gmail.com (Bernard Amade) Date: Mon, 5 Mar 2018 10:54:41 +0100 Subject: Main class in jar Message-ID: Hello all the doc in https://docs.oracle.com/javase/9/docs/specs/jar/jar.html lists for Main-Class entry in jars: "attribute defined for stand-alone applications: This attribute is used by stand-alone applications that are bundled into executable jar files which can be invoked by the java runtime directly by running "java -jar x.jar". apparently it means that you should not create a modular jar with a manifest template containing a Main-Class entry (apparently if you run the app with command "java -module-path dir -m com.thing.app" with the generated com.thing.app.jar the main class will not be found : you get a message saying you've got no Mainclass atrribute) but if you generate the jar with a manifest template with no Main-Class entry and instead use the --main-class option then the main class will be found. funnily the MANIFEST-MF will then have the same Main-Class entry! so what is the difference? (btw I suggest more docs that differentiate clearly what to do 1) with non-modular jars 2) with modular jars .. ... or is it I can't read the doc properly?) thanks Bear From Alan.Bateman at oracle.com Mon Mar 5 10:25:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 5 Mar 2018 10:25:08 +0000 Subject: Main class in jar In-Reply-To: References: Message-ID: On 05/03/2018 09:54, Bernard Amade wrote: > Hello all > the doc in https://docs.oracle.com/javase/9/docs/specs/jar/jar.html > lists for Main-Class entry in jars: > "attribute defined for stand-alone applications: This attribute is used by stand-alone applications that are bundled into executable jar files which can be invoked by the java runtime directly by running "java -jar x.jar". > apparently it means that you should not create a modular jar with a manifest template containing a Main-Class entry > (apparently if you run the app with command "java -module-path dir -m com.thing.app" with the generated com.thing.app.jar the main class will not be found : you get a message saying you've got no Mainclass atrribute) > but if you generate the jar with a manifest template with no Main-Class entry and instead use the --main-class option then the main class will be found. > funnily the MANIFEST-MF will then have the same Main-Class entry! > so what is the difference? The error message with JDK 9 and JDK 10 has a typo which may be causing confusion here. The typo has been fixed for JDK 11 via JDK-8193819 to correctly say "ModuleMainClass" attribute. Note that this is a class file attribute, not an attribute that you put in the main manifest of a JAR file. The `jar` tool will add both when you specify the main class with `--main-class` or `-e`. To summarize: 1. `jar -jar app.jar` is the equivalent of `java -cp app.jar ` where is value of the Main-Class attribute in the main manifest of the JAR file. 2. `java [--module-path ] -m ` is the way to run a modular application on the command line. The app module must have a ModuleMainClass class file attribute, something the `jar` tool will do for you (Maven plugins and IDEs should do the same but might not be all there yet).? The alternative is `-m app/` which you specify both the initial/main module and also its main class. 3.? There is no support for executing modular applications with `java -jar`. -Alan From Alan.Bateman at oracle.com Mon Mar 5 11:05:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 5 Mar 2018 11:05:22 +0000 Subject: Main class in jar In-Reply-To: References: Message-ID: <99e279be-36ee-7931-4299-89b19edfdbcf@oracle.com> On 05/03/2018 10:43, Bernard Amade wrote: > thanks for your help > > just curious about this internal information: > > --------------------------------------------------- > "ModuleMainClass" attribute. Note that this is a class file attribute, not an attribute that you put in the main manifest of a JAR file. The `jar` tool will add both when you specify the main class with `--main-class` or `-e`. >> The app module must have a ModuleMainClass class file attribute, something the `jar` tool will do for you > ---------------------------------- > where does this "ModuleMainClass" attribute lies internally? > (I've tried to decompile almost every file but have not found it) It's an attribute in the module-info.class. You can use `javap` to see all the details if you want, e.g. ? javap --module-path app.jar -m app -v module-info The `jar` tool might be easier for your audience, e.g. ? jar --file=app.jar --describe-module > : > > PS: if I understand things correctly the reason for not having an autonomous jar (like in java -jar) is that > starting a modular application from jars needs a lot of specific command-line options .... > Do we really have to stick to shell scripts to start an app? (when we do not go for a jlink generated image) If the initial module is not in the run-time image then you have to specify the module path so that the module can be found. Re-purposing `java -jar` was discussed here but for the initial release at least, `java -jar ...` will put the JAR file on the class path as it has always done. It's also related to the #MultiModuleExecutableJARs issue where there is interest in packaging an initial module, along transitive dependences, in a single JAR file. So I expect this area will be revisited at some point. -Alan From greggwon at cox.net Mon Mar 5 13:58:23 2018 From: greggwon at cox.net (Gregg Wonderly) Date: Mon, 5 Mar 2018 07:58:23 -0600 Subject: Main class in jar In-Reply-To: References: Message-ID: <906ECB5F-4942-4B24-84EB-E846A36671CE@cox.net> Will java -jar note that the argument is a module and help the user understand how to invoke it, or will it just complain about a missing main-class: attribute? From a practical perspective, why does it matter and demand a different command line? What happens when you double click on a module? Gregg Sent from my iPad > On Mar 5, 2018, at 4:25 AM, Alan Bateman wrote: > >> On 05/03/2018 09:54, Bernard Amade wrote: >> Hello all >> the doc in https://docs.oracle.com/javase/9/docs/specs/jar/jar.html >> lists for Main-Class entry in jars: >> "attribute defined for stand-alone applications: This attribute is used by stand-alone applications that are bundled into executable jar files which can be invoked by the java runtime directly by running "java -jar x.jar". >> apparently it means that you should not create a modular jar with a manifest template containing a Main-Class entry >> (apparently if you run the app with command "java -module-path dir -m com.thing.app" with the generated com.thing.app.jar the main class will not be found : you get a message saying you've got no Mainclass atrribute) >> but if you generate the jar with a manifest template with no Main-Class entry and instead use the --main-class option then the main class will be found. >> funnily the MANIFEST-MF will then have the same Main-Class entry! >> so what is the difference? > The error message with JDK 9 and JDK 10 has a typo which may be causing confusion here. The typo has been fixed for JDK 11 via JDK-8193819 to correctly say "ModuleMainClass" attribute. Note that this is a class file attribute, not an attribute that you put in the main manifest of a JAR file. The `jar` tool will add both when you specify the main class with `--main-class` or `-e`. > > To summarize: > > 1. `jar -jar app.jar` is the equivalent of `java -cp app.jar ` where is value of the Main-Class attribute in the main manifest of the JAR file. > > 2. `java [--module-path ] -m ` is the way to run a modular application on the command line. The app module must have a ModuleMainClass class file attribute, something the `jar` tool will do for you (Maven plugins and IDEs should do the same but might not be all there yet). The alternative is `-m app/` which you specify both the initial/main module and also its main class. > > 3. There is no support for executing modular applications with `java -jar`. > > -Alan From Alan.Bateman at oracle.com Mon Mar 5 15:40:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 5 Mar 2018 15:40:29 +0000 Subject: Main class in jar In-Reply-To: <906ECB5F-4942-4B24-84EB-E846A36671CE@cox.net> References: <906ECB5F-4942-4B24-84EB-E846A36671CE@cox.net> Message-ID: On 05/03/2018 13:58, Gregg Wonderly wrote: > Will java -jar note that the argument is a module and help the user understand how to invoke it, or will it just complain about a missing main-class: attribute? From a practical perspective, why does it matter and demand a different command line? What happens when you double click on a module? > `java -jar` has not changed,? it will print an error to say that there is "no main manifest attribute" as it has always done. If your modules aren't in the run-time image then you have to specify a module path to the java launcher. This is different to `java -jar` where you've packaged everything into a single JAR file (ignoring Class-Path in the this discussion for now). As I said, there is a carry over issue from JSR 376 to examine the topic of executable modular "uber JARs" so I expect it will be re-examined. I assume by "double on a module" then you mean a JAR file. In that case it depends on how your desktop is configured. Some users may have it configured to run "javaw.exe" with `-jar %1` as the parameters, this would be the equivalent of typing `java -jar ...` on the command line. -Alan From greggwon at cox.net Mon Mar 5 22:30:42 2018 From: greggwon at cox.net (Gregg Wonderly) Date: Mon, 5 Mar 2018 16:30:42 -0600 Subject: Main class in jar In-Reply-To: References: <906ECB5F-4942-4B24-84EB-E846A36671CE@cox.net> Message-ID: <116F5E4D-9717-4C52-A3DB-6864741A53F3@cox.net> But since jar vs module both require a manifest entry or command-line argument to specify the exact thing to ?run?, can?t they both inspect the opportunity to act like the other form of command line, and then do the right thing? The problem is that ?uber JARs? (which I have used for decades at this point), used as ?applications?, being doubled clicked on, will require there to be this inspection and intelligent execution. I feel its really important to make this seamless so that people who really do use java as a desktop application environment can provide a single jar to allow users to use the application. Installers are just not something that we need to start the path to have to support cross platform? Gregg > On Mar 5, 2018, at 9:40 AM, Alan Bateman wrote: > > On 05/03/2018 13:58, Gregg Wonderly wrote: >> Will java -jar note that the argument is a module and help the user understand how to invoke it, or will it just complain about a missing main-class: attribute? From a practical perspective, why does it matter and demand a different command line? What happens when you double click on a module? >> > `java -jar` has not changed, it will print an error to say that there is "no main manifest attribute" as it has always done. > > If your modules aren't in the run-time image then you have to specify a module path to the java launcher. This is different to `java -jar` where you've packaged everything into a single JAR file (ignoring Class-Path in the this discussion for now). As I said, there is a carry over issue from JSR 376 to examine the topic of executable modular "uber JARs" so I expect it will be re-examined. > > I assume by "double on a module" then you mean a JAR file. In that case it depends on how your desktop is configured. Some users may have it configured to run "javaw.exe" with `-jar %1` as the parameters, this would be the equivalent of typing `java -jar ...` on the command line. > > -Alan From guillaume at hibernate.org Tue Mar 6 13:52:08 2018 From: guillaume at hibernate.org (Guillaume Smet) Date: Tue, 6 Mar 2018 14:52:08 +0100 Subject: java.beans package in java.desktop module Message-ID: Hi, (Previously sent via an unsubscribed email address, sorry about that) The java.beans package is part of the java.desktop module which is a bit unfortunate as the package contains quite a few classes useful to manipulate beans and dragging all the desktop classes with them is far from ideal. Typically, we have: - javax.el which uses java.beans.FeatureDescriptor (javax.el is the standard EL implementation used by Bean Validation) - Hibernate ORM which uses java.beans.Introspector (and thus BeanInfo and so on) - Is there a plan to get java.beans out of java.desktop? Or should we avoid its usage? In our case, we can deal with the Hibernate ORM part but, for javax.el, it might be a bit more complicated as FeatureDescriptor is unfortunately included in specified APIs. Thanks for the feedback. -- Guillaume From scolebourne at joda.org Tue Mar 6 14:29:48 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 6 Mar 2018 14:29:48 +0000 Subject: java.beans package in java.desktop module In-Reply-To: References: Message-ID: It had been my hope that we might see a replacement for the java.beans package. I drew up a rough prototype here: https://github.com/jodastephen/property-alliance based on previous work in Joda-Beans. More info here: http://jodastephen.github.io/property-alliance/ If an interface-based design were adopted, it could be implemented by the existing JavaBeans code and also by future language changes such as records (data classes), while the API could be used far more broadly, such as in Hibernate or EL. Sadly, I haven't had the time or energy to progress this, but the need is there. Stephen On 6 March 2018 at 13:52, Guillaume Smet wrote: > The java.beans package is part of the java.desktop module which is a bit > unfortunate as the package contains quite a few classes useful to > manipulate beans and dragging all the desktop classes with them is far from > ideal. > > Typically, we have: > - javax.el which uses java.beans.FeatureDescriptor (javax.el is the > standard EL implementation used by Bean Validation) > - Hibernate ORM which uses java.beans.Introspector (and thus BeanInfo and > so on) > - > > Is there a plan to get java.beans out of java.desktop? Or should we avoid > its usage? > > In our case, we can deal with the Hibernate ORM part but, for javax.el, it > might be a bit more complicated as FeatureDescriptor is unfortunately > included in specified APIs. > > Thanks for the feedback. From Alan.Bateman at oracle.com Tue Mar 6 14:51:30 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 6 Mar 2018 14:51:30 +0000 Subject: java.beans package in java.desktop module In-Reply-To: References: Message-ID: <43293e39-be00-b366-86df-ef9d56ddc63d@oracle.com> On 06/03/2018 13:52, Guillaume Smet wrote: > Hi, > > (Previously sent via an unsubscribed email address, sorry about that) > > The java.beans package is part of the java.desktop module which is a bit > unfortunate as the package contains quite a few classes useful to > manipulate beans and dragging all the desktop classes with them is far from > ideal. > > Typically, we have: > - javax.el which uses java.beans.FeatureDescriptor (javax.el is the > standard EL implementation used by Bean Validation) > - Hibernate ORM which uses java.beans.Introspector (and thus BeanInfo and > so on) > - > > Is there a plan to get java.beans out of java.desktop? Or should we avoid > its usage? > > In our case, we can deal with the Hibernate ORM part but, for javax.el, it > might be a bit more complicated as FeatureDescriptor is unfortunately > included in specified APIs. > The java.beans package is in the desktop module because it contains several APIs that tie it to AWT and other areas of java.desktop (Beans.getIcon for example). The mistake 20 years ago was to put design time APIs for beans in the same package as the APIs to use them at run-time. Countless hours went into previous releases looking into options but all routes involve pain and incompatible changes. The deprecation of the applet API (JEP 289) helps a little bit as it opens the potential for the APIs tied to applets (Beans.instaniate for example) to be removed but the substantive issue remains. There aren't any concrete proposals on the table at this time but it is something that will need to be looked at again. I'm curious why you are bringing it up. Are you looking to deploy on a run-time image that doesn't have the java.desktop module? -Alan. From guillaume at hibernate.org Tue Mar 6 16:43:27 2018 From: guillaume at hibernate.org (Guillaume Smet) Date: Tue, 6 Mar 2018 17:43:27 +0100 Subject: java.beans package in java.desktop module In-Reply-To: <43293e39-be00-b366-86df-ef9d56ddc63d@oracle.com> References: <43293e39-be00-b366-86df-ef9d56ddc63d@oracle.com> Message-ID: Hi Alan, On Tue, Mar 6, 2018 at 3:51 PM, Alan Bateman wrote: > There aren't any concrete proposals on the table at this time but it is > something that will need to be looked at again. > OK. That's what I suspected, unfortunately. I think we will probably be able to get rid of our Introspector usage at some point but the javax.el case is definitely something that's bugging me. > I'm curious why you are bringing it up. Are you looking to deploy on a > run-time image that doesn't have the java.desktop module? > Well, I'm not looking at anything right now. We just wanted to not have our Hibernate libraries (in this case Hibernate ORM via Introspector and Hibernate Validator via javax.el) tied up to the java.desktop module so that our users could try to deploy them on a runtime image that doesn't have this very module. Getting rid of the java.desktop module is IMHO one of the nice thing of the new modular system. -- Guillaume From guillaume at hibernate.org Tue Mar 6 17:45:21 2018 From: guillaume at hibernate.org (Guillaume Smet) Date: Tue, 6 Mar 2018 18:45:21 +0100 Subject: java.beans package in java.desktop module In-Reply-To: References: Message-ID: Hi Stephen, On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne wrote: > It had been my hope that we might see a replacement for the java.beans > package. I drew up a rough prototype here: > https://github.com/jodastephen/property-alliance > based on previous work in Joda-Beans. More info here: > http://jodastephen.github.io/property-alliance/ > > If an interface-based design were adopted, it could be implemented by > the existing JavaBeans code and also by future language changes such > as records (data classes), while the API could be used far more > broadly, such as in Hibernate or EL. > > Sadly, I haven't had the time or energy to progress this, but the need is > there. Thanks for sharing. It indeed sound like something worth pursuing. Having a new API in the JDK indeed seems the only way out for the java.beans issue. -- Guillaume From forax at univ-mlv.fr Wed Mar 7 11:21:24 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 7 Mar 2018 12:21:24 +0100 (CET) Subject: java.beans package in java.desktop module In-Reply-To: References: Message-ID: <1511880147.1119953.1520421684465.JavaMail.zimbra@u-pem.fr> As Stephen said, with the introduction of the Pattern Matching in the near future, an API to extract the values from an object (the de-constructor API) or at least from a record object will have to be created, but it may be based on method handles, so perhaps not using a direct interface. R?mi ----- Mail original ----- > De: "Guillaume Smet" > ?: "Stephen Colebourne" > Cc: "jigsaw-dev" > Envoy?: Mardi 6 Mars 2018 18:45:21 > Objet: Re: java.beans package in java.desktop module > Hi Stephen, > > On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne > wrote: > >> It had been my hope that we might see a replacement for the java.beans >> package. I drew up a rough prototype here: >> https://github.com/jodastephen/property-alliance >> based on previous work in Joda-Beans. More info here: >> http://jodastephen.github.io/property-alliance/ >> >> If an interface-based design were adopted, it could be implemented by >> the existing JavaBeans code and also by future language changes such >> as records (data classes), while the API could be used far more >> broadly, such as in Hibernate or EL. >> >> Sadly, I haven't had the time or energy to progress this, but the need is >> there. > > > Thanks for sharing. It indeed sound like something worth pursuing. > > Having a new API in the JDK indeed seems the only way out for the > java.beans issue. > > -- > Guillaume From scolebourne at joda.org Wed Mar 7 12:11:07 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Mar 2018 12:11:07 +0000 Subject: ClassLoader.getResources(String) In-Reply-To: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> References: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> Message-ID: Following up on this, it does feel like the use case is now simply not possible. I have a similar problem with ClassLoader.getResources(String) in threeten-extra. https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202 The ThreeTen-Extra project defines a config file org/threeten/extra/scale/LeapSecond.txt. The code uses ClassLoader.getResources(String) to find the latest version of the file, which may be in the threeten-extra jar file, or in any jar file that uses threeten-extra.jar. ie. to replace the version from threeten-extra.jar, a user simply has to add a file with the same name/package to their jar file. threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt application.jar also contains org/threeten/extra/scale/LeapSecond.txt Under JPMS this fails, as the resource cannot be located in org/threeten/extra/scale in a different jar file. But this appears makes the whole design impossible to make work with JPMS. The code in threeten-extra.jar cannot possibly know about the package names of the application.jar, so there is no way for it to find the config file. There seem to be only two solutions to this - ServiceLoader, but that is for code, not config files - forcing the application to manually register their config file Both of these provide a markedly worse outcome. Am I missing something? Stephen On 7 February 2018 at 20:11, Alan Bateman wrote: > On 07/02/2018 16:56, Stephen Colebourne wrote: >> >> : >> I was using maven to create a jar-with-dependencies file, so I could >> use jlink. With all the code in one jar file, there shouldn't be any >> access barriers to worry about. >> >> ClassLoader.getResources(String) worked just fine until Java 9. The >> two APIs are not comparable - the ClassLoader one returns all URLs >> found, whereas the Class one returns just one URL. Switching API would >> change behaviour. > > ClassLoader.getResources searches the class path as it did in JDK 9 and > older, it it just can't locate non-".class" resources in modules when they > are encapsulated. Class loaders are oblivious as to who is ultimately > attempting to load a class or locate a resource (the initiating and defining > loader can be different, they can many class loaders in the delegation > chain). > > With the uber modular JAR scenario then all classes for several libraries > are in the same module. This means that the names of resources in that > module are unique. If several libraries have the same resource then I assume > you drop all but one when you create this uber JAR (or maybe you are merging > some of the configuration files, I can't tell). So I assume you could change > this code to use Class.getResource and it will locate at-most-one resource > with a specific name. > > To do a proper migration means re-examining ResourceConfig of course. Using > services is likely to be a lot cleaner and more robust than scanning for > configuration files. > > -Alan From scolebourne at joda.org Wed Mar 7 12:23:10 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Mar 2018 12:23:10 +0000 Subject: java.beans package in java.desktop module In-Reply-To: <1511880147.1119953.1520421684465.JavaMail.zimbra@u-pem.fr> References: <1511880147.1119953.1520421684465.JavaMail.zimbra@u-pem.fr> Message-ID: What is needed is an abstraction that can work for all sorts of data-like classes - classic JavaBeans - records - value types - HashMaps - JSON objects - etc Its not rocket science as an API, but has been needed for many years as so many projects have this code duplicated (leading to lots of subtle differences). The API cannot be based on method handles, as in the HashMap case there is no property-specific method to call. But there is no reason why the implementation of the interface for records could not use method handles internally. Stephen On 7 March 2018 at 11:21, Remi Forax wrote: > As Stephen said, with the introduction of the Pattern Matching in the near future, an API to extract the values from an object (the de-constructor API) or at least from a record object will have to be created, but it may be based on method handles, so perhaps not using a direct interface. > > R?mi > > ----- Mail original ----- >> De: "Guillaume Smet" >> ?: "Stephen Colebourne" >> Cc: "jigsaw-dev" >> Envoy?: Mardi 6 Mars 2018 18:45:21 >> Objet: Re: java.beans package in java.desktop module > >> Hi Stephen, >> >> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne >> wrote: >> >>> It had been my hope that we might see a replacement for the java.beans >>> package. I drew up a rough prototype here: >>> https://github.com/jodastephen/property-alliance >>> based on previous work in Joda-Beans. More info here: >>> http://jodastephen.github.io/property-alliance/ >>> >>> If an interface-based design were adopted, it could be implemented by >>> the existing JavaBeans code and also by future language changes such >>> as records (data classes), while the API could be used far more >>> broadly, such as in Hibernate or EL. >>> >>> Sadly, I haven't had the time or energy to progress this, but the need is >>> there. >> >> >> Thanks for sharing. It indeed sound like something worth pursuing. >> >> Having a new API in the JDK indeed seems the only way out for the >> java.beans issue. >> >> -- >> Guillaume From forax at univ-mlv.fr Wed Mar 7 12:55:39 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 7 Mar 2018 13:55:39 +0100 (CET) Subject: java.beans package in java.desktop module In-Reply-To: References: <1511880147.1119953.1520421684465.JavaMail.zimbra@u-pem.fr> Message-ID: <2044726300.1163671.1520427339159.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stephen Colebourne" > ?: "jigsaw-dev" > Envoy?: Mercredi 7 Mars 2018 13:23:10 > Objet: Re: java.beans package in java.desktop module > What is needed is an abstraction that can work for all sorts of > data-like classes > > - classic JavaBeans > - records > - value types > - HashMaps > - JSON objects > - etc > > Its not rocket science as an API, but has been needed for many years > as so many projects have this code duplicated (leading to lots of > subtle differences). > > The API cannot be based on method handles, as in the HashMap case > there is no property-specific method to call. you can bind [1] (do partial evaluation of) Map::get with the property name, > But there is no reason why the implementation of the interface for records could not use > method handles internally. apart if valhalla generics are around, such interface will require to box and unbox values. > > Stephen R?mi > > > On 7 March 2018 at 11:21, Remi Forax wrote: >> As Stephen said, with the introduction of the Pattern Matching in the near >> future, an API to extract the values from an object (the de-constructor API) or >> at least from a record object will have to be created, but it may be based on >> method handles, so perhaps not using a direct interface. >> >> R?mi >> >> ----- Mail original ----- >>> De: "Guillaume Smet" >>> ?: "Stephen Colebourne" >>> Cc: "jigsaw-dev" >>> Envoy?: Mardi 6 Mars 2018 18:45:21 >>> Objet: Re: java.beans package in java.desktop module >> >>> Hi Stephen, >>> >>> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne >>> wrote: >>> >>>> It had been my hope that we might see a replacement for the java.beans >>>> package. I drew up a rough prototype here: >>>> https://github.com/jodastephen/property-alliance >>>> based on previous work in Joda-Beans. More info here: >>>> http://jodastephen.github.io/property-alliance/ >>>> >>>> If an interface-based design were adopted, it could be implemented by >>>> the existing JavaBeans code and also by future language changes such >>>> as records (data classes), while the API could be used far more >>>> broadly, such as in Hibernate or EL. >>>> >>>> Sadly, I haven't had the time or energy to progress this, but the need is >>>> there. >>> >>> >>> Thanks for sharing. It indeed sound like something worth pursuing. >>> >>> Having a new API in the JDK indeed seems the only way out for the >>> java.beans issue. >>> >>> -- > >> Guillaume From Alan.Bateman at oracle.com Wed Mar 7 12:59:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Mar 2018 12:59:14 +0000 Subject: ClassLoader.getResources(String) In-Reply-To: References: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> Message-ID: <46ad8cc5-6bb7-3a08-37a3-3ab8b11d4b66@oracle.com> On 07/03/2018 12:11, Stephen Colebourne wrote: > Following up on this, it does feel like the use case is now simply not > possible. I have a similar problem with > ClassLoader.getResources(String) in threeten-extra. > > https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202 > > The ThreeTen-Extra project defines a config file > org/threeten/extra/scale/LeapSecond.txt. The code uses > ClassLoader.getResources(String) to find the latest version of the > file, which may be in the threeten-extra jar file, or in any jar file > that uses threeten-extra.jar. ie. to replace the version from > threeten-extra.jar, a user simply has to add a file with the same > name/package to their jar file. > > threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt > application.jar also contains org/threeten/extra/scale/LeapSecond.txt > > Under JPMS this fails, as the resource cannot be located in > org/threeten/extra/scale in a different jar file. But this appears > makes the whole design impossible to make work with JPMS. > > The code in threeten-extra.jar cannot possibly know about the package > names of the application.jar, so there is no way for it to find the > config file. > > There seem to be only two solutions to this > - ServiceLoader, but that is for code, not config files > - forcing the application to manually register their config file > > Both of these provide a markedly worse outcome. > > Am I missing something? > Resources can't be both encapsulated and not encapsulated at the same time. If a module has a resource in a package that is intended to be located by code in other modules using ClassLoader getResourceXXX then it has to open the package. In the above, then I assume the main issue isn't resource encapsulation, it's that you've got two modules on the application module class containing the same package so they can't both be mapped to the application class loader. You've dismissed services but I would expect it to provide a nice solution. The service interface might be very simple, something like: public interface LeapSecondDataProvider { ??? LeapSecondData data(); } or better still, define methods that allow SystemUTCRules select the right version of the leap second data. Applications that ship their own leap data second would ship an implementation of this class. Yes, it's different to searching the class file for configuration files but a lot more reliable. -Alan From scolebourne at joda.org Wed Mar 7 15:14:11 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Mar 2018 15:14:11 +0000 Subject: ClassLoader.getResources(String) In-Reply-To: <46ad8cc5-6bb7-3a08-37a3-3ab8b11d4b66@oracle.com> References: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> <46ad8cc5-6bb7-3a08-37a3-3ab8b11d4b66@oracle.com> Message-ID: On 7 March 2018 at 12:59, Alan Bateman wrote: > You've dismissed services but I would expect it to provide a nice solution. > The service interface might be very simple, something like: > > public interface LeapSecondDataProvider { > LeapSecondData data(); > } Configuration and code are two very different things. Asking projects and end users to write code for something that should be config is a huge no-no. My view is that JPMS has made using configuration files, especially for libraries, a lot harder. This is a step back in usability. Just so we are clear, for leap seconds I will now have to ask users to manually register them using an API where previously they just added a file. But for OpenGamma Strata, the configuration files are much more complex and certainly unsuited to be code, even if the backwards compatibility issues were acceptable. (This is a pattern I've used for configuration for many years) Effectively what is needed is another way for a library to be informed of the presence of the calling application. One possible solution to this would be to allow users to write module initialization code in module-info.java. Then an application coder would have a solid reliable place to put code that registers the additional configuration files with the low-level library. Something like: module com.foo.app { requires org.threeten.extra; init(ModuleInitContext context) { UtcRules.registerLeapSecondFile("/com/foo/app/LeapSeconds.txt"); } } PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't know whether it will run as a named module or on the classpath, I have to duplicate the service loader configuration - once in META-INF/services and once in module-info.java, which is horrible. It also means that the provide() static method is a feature that cannot be used by libraries. Stephen From org.openjdk at io7m.com Wed Mar 7 15:34:01 2018 From: org.openjdk at io7m.com (Mark Raynsford) Date: Wed, 7 Mar 2018 15:34:01 +0000 Subject: ClassLoader.getResources(String) In-Reply-To: References: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> <46ad8cc5-6bb7-3a08-37a3-3ab8b11d4b66@oracle.com> Message-ID: <20180307153401.5962015b@copperhead.int.arc7.info> On 2018-03-07T15:14:11 +0000 Stephen Colebourne wrote: > > Effectively what is needed is another way for a library to be informed > of the presence of the calling application. One possible solution to > this would be to allow users to write module initialization code in > module-info.java. Then an application coder would have a solid > reliable place to put code that registers the additional configuration > files with the low-level library. There's a pattern I've seen used in OSGi that I've considered adapting for my own use (exposing resource-only modules that don't contain any code and yet need to be able to cause some other piece of code to instantiate services on behalf of the module): They call it the "extender pattern". https://dzone.com/articles/osgi-42-extender-pattern-and Briefly, what happens is that you subscribe to an interface that tells you when bundles (OSGi terminology for artifacts containing modules, more or less) are added to or removed from the system. In Jigsaw, this would probably equate to publishing some sort of event that can be observed whenever someone creates a new module layer. When you get notified that a bundle has been added, you can scan the manifest of the bundle (via the standard jar manifest APIs if you like) and can then, for example, look for manifest fields that tell you where in the jar file to find application-specific config files. X-My-Extra-Config-File: /com/example/config.xml The listening party can then read the config file, instantiate services as necessary, etc. It'd need to be handled in a way that ensured that you don't essentially race the module resolution code; Just because your code didn't get a chance to subscribe to "module became available" events until after they'd appeared doesn't mean you should miss the events. OSGi does handle this (it's written into the spec), but I don't know quite how the implementations handle it. -- Mark Raynsford | http://www.io7m.com From Alan.Bateman at oracle.com Wed Mar 7 15:59:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 7 Mar 2018 15:59:11 +0000 Subject: ClassLoader.getResources(String) In-Reply-To: References: <0d7c3dcd-8e19-5b4e-fe18-79d80652a7a2@oracle.com> <46ad8cc5-6bb7-3a08-37a3-3ab8b11d4b66@oracle.com> Message-ID: On 07/03/2018 15:14, Stephen Colebourne wrote: > : > Configuration and code are two very different things. Asking projects > and end users to write code for something that should be config is a > huge no-no. > > My view is that JPMS has made using configuration files, especially > for libraries, a lot harder. This is a step back in usability. Just so > we are clear, for leap seconds I will now have to ask users to > manually register them using an API where previously they just added a > file. But for OpenGamma Strata, the configuration files are much more > complex and certainly unsuited to be code, even if the backwards > compatibility issues were acceptable. (This is a pattern I've used for > configuration for many years) It's usually cleaner to encapsulate that configuration but if you don't want to change anything then you can continue to use ClassLoader.getResources to search for resources as it works exactly as it did before. Also if you move the configuration file to somewhere like META-INF/config then it can never be encapsulated. > : > > > PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't > know whether it will run as a named module or on the classpath, I have > to duplicate the service loader configuration - once in > META-INF/services and once in module-info.java, which is horrible. > Tooling should be able help with cases where you are creating a library that may be deployed on the class path in some environments and the module path in others. The `jar` tool does some sanity check in this area but it could do more. -Alan From bear.amade at gmail.com Tue Mar 13 09:23:52 2018 From: bear.amade at gmail.com (Bernard Amade) Date: Tue, 13 Mar 2018 10:23:52 +0100 Subject: why is modularity changing the rules of inheritance? Message-ID: <20E28D4A-0A87-4DDE-BAD1-F9991F67E6A0@gmail.com> Hello all (OOps! wrote "Hell all" ) just curious to read about the rationale behind this behaviour: if you have a code in moduleB that inherits from code in moduleA, modularity wise the inherited methods behave as if they were invoked from moduleA (not from moduleB). see an example at the end of this post. Na?vely found this behaviour *very bad* but I suppose that if it were not the case then the situation could be worse... so I would like to read the rationale behind this. thanks code of a service definition in moduleA: ----------------------------------------------- public abstract class PropertiesProvider { public Properties get(String domain) { Class clazz =this.getClass(); System.out.println(" CLASS " +clazz); try { Object obj = clazz.getConstructor().newInstance(); System.out.println("Object " + obj); } catch (Exception exc) { exc.printStackTrace(); } InputStream is = clazz.getResourceAsStream(domain) ; if (is != null) { Properties props = new Properties(); try { props.load(is); return props; } catch (IOException e) { System.getLogger("").log(System.Logger.Level.WARNING,e) ; } } and so on ... now if in module B you just create a Service by inheriting from this class: - the className is correct (an instance of the class has been created for the service) - you can't invoke newInstance from that very class!(if the class is not opened or exported) - you can't get the Resource (or other files) unless they are explicilty opened. if you copy this code in the class implementing the service everything just works fine so there is a difference between the same code being explicitly in the class or inherited! (got a headache trying to find something to say for people learning Java) From Alan.Bateman at oracle.com Tue Mar 13 10:12:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Mar 2018 10:12:22 +0000 Subject: why is modularity changing the rules of inheritance? In-Reply-To: <20E28D4A-0A87-4DDE-BAD1-F9991F67E6A0@gmail.com> References: <20E28D4A-0A87-4DDE-BAD1-F9991F67E6A0@gmail.com> Message-ID: On 13/03/2018 09:23, Bernard Amade wrote: > now if in module B you just create a Service by inheriting from this class: > - the className is correct (an instance of the class has been created for the service) > - you can't invoke newInstance from that very class!(if the class is not opened or exported) Constructor::newInstance has always been caller sensitive. If I read the example correctly, you've got code in class A trying to create an instance of class B. B is not accessible to A, it doesn't matter that B extends A. The issue you are running into isn't new with modules - you'll see the same IllegalAccessException with the class path for the case that B is not public (and in a different package to A). > - you can't get the Resource (or other files) unless they are explicilty opened. > if you copy this code in the class implementing the service everything just works fine > so there is a difference between the same code being explicitly in the class or inherited! The Class.getResourceXXX methods are also caller sensitive so the example is code in class A trying to locate a resource in its own module. If you maintain both A and B then it would be simpler if B were to call the method in the super class with a connection or URL to the resource. -Alan. From Rony.Flatscher at wu.ac.at Tue Mar 13 11:54:26 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 13 Mar 2018 12:54:26 +0100 Subject: Strange observation: MethodHandle.invokeWithArguments() would not work, whereas Method.invoke() would with the very same arguments In-Reply-To: References: Message-ID: Just to conclude: this is something that has nothing to do with jigsaw per se. If interested you could look further to mlvm-dev ("Da Vinci Machine Project"), here two respective links: and a pure Java example pointing at the problem: . ---rony On 12.02.2018 20:59, Rony G. Flatscher wrote: > In the process of adapting pure reflective code (a Rexx-Java bridge) to use MethodHandles on Java 9 > instead, everything seems to be working out so far. > > In principle all invocations on the Java side are carried out by first using java.lang.reflect > (Field, Method, Constructor) using the supplied arguments (if the arguments can be coerced to the > respective parameterTypes it gets selected for invocation)? and if a candidate is found an > appropriate MethodHandle gets created, which then gets used to invoke the operation supplying the > coerced arguments, if any. Over the weekend I finalized the basic changes and started to test > against a set of sample/demo applications. > > --- > > While testing a rather complex one (an adaption of the JavaFX address book example enhanced with a > BarChart, [1]), that exhibits a very strange behavior: when setting the values for the CategoryAxis > supplying an ObservableList of the month names in the current Locale, using a MethodHandle and > invoking it with invokeWithArguments() would yield (debug output): > > // // // RexxReflectJava9.processMethod(), ARRIVED: -> [INVOKE], tmpMethod=[public final void > javafx.scene.chart.CategoryAxis.setCategories(javafx.collections.ObservableList)]: > method=[SETCATEGORIES] in > object=[rru.rexxArgs[1]="javafx.scene.chart.CategoryAxis at 83278e1"/rru.bean="CategoryAxis[id=xAxis, > styleClass=axis]"] > > // // // RexxReflectJava9.processMethod(), > coercedArgs=[[[Ljava.lang.String;@57cfe770]].getClass().toString()=class [Ljava.lang.Object;, > ??? ??? ?parameterTypes=[interface > javafx.collections.ObservableList].getClass().toString()=class [Ljava.lang.Class;:, > ??? ??? ?rru.funcArgs=[[[Ljava.lang.String;@57cfe770]].getClass().toString()=class > [Ljava.lang.Object; > > // // :( RexxReflectJava9.processMethod(), MethodType for Method [public final void > javafx.scene.chart.CategoryAxis.setCategories(javafx.collections.ObservableList)]: > "(ObservableList)void" > > // // :( RexxReflectJava9.processMethod(): INSTANCE, mh.bindTo("CategoryAxis[id=xAxis, > styleClass=axis]/class javafx.scene.chart.CategoryAxis").invokeWithArguments(...) > > // :) :) RexxReflectJava9.processMethod(), MethodHandle > "MethodHandle(CategoryAxis,ObservableList)void" invocation caused a Throwable: > java.lang.ClassCastException: java.base/[Ljava.lang.String; cannot be cast to > java.base/java.lang.String > > > The supplied ObservableList argument represents the? month names and was created with the help of > "javafx.collections.FXCollections.observableList()" and then using its "addAll(monthNames)" method > to add the String array values returned by DateFormatSymbols.getMonths() to the list. > > The supplied argument array "rru.funcArgs" will be coerced according to the reflected > "parameterTypes" array yielding the "coercedArgs" array; using java.util.Arrays.deepToString() gives: > > // // // RexxReflectJava9.processMethod(), > coercedArgs=[[[Ljava.lang.String;@57cfe770]].getClass().toString()=class [Ljava.lang.Object;, > ??? ??? ?parameterTypes=[interface > javafx.collections.ObservableList].getClass().toString()=class [Ljava.lang.Class;:, > ??? ??? ?rru.funcArgs=[[[Ljava.lang.String;@57cfe770]].getClass().toString()=class > [Ljava.lang.Object; > > --- > > The story is much longer but after quite long debugging sessions, I turned on reflective invoke via > tmpMethod instead of invoking the corresponding MethodHandle, which (surprisingly) works. > > Then, in the next step doing the same invocation via the corresponding MethodHandle immediately > after the reflective invocation, allows that invocation to execute successfully as well! > > Please note, the supplied coerced argument is in both cases the same! Coercion occurs according to > the "parameterTypes" returned by java.lang.reflect.Method which also is used for creating the > MethodType in order to use a publicLookup.findVirtual(...). Or with other words: the coerced > argument will be identical for both invocation types! > > > Another strange observation in the success case is as follows: when using reflective invocation by > default (and then only invoking the MethodHandle in the special case that the method "setCategories" > is to be executed) the coerced argument supplied to java.util.Arrays.deepToString() will list the > monthnames: > > // // // RexxReflectJava9.processMethod(), ARRIVED: -> [INVOKE], tmpMethod=[public final void > javafx.scene.chart.CategoryAxis.setCategories(javafx.collections.ObservableList)]: > method=[SETCATEGORIES] in > object=[rru.rexxArgs[1]="javafx.scene.chart.CategoryAxis at 2d809949"/rru.bean="CategoryAxis[id=xAxis, > styleClass=axis]"] > > // // // RexxReflectJava9.processMethod(), coercedArgs=[[January, February, March, April, May, > June, July, August, September, October, November, December, ]].getClass().toString()=class > [Ljava.lang.Object;, > ??? ??? ?parameterTypes=[interface > javafx.collections.ObservableList].getClass().toString()=class [Ljava.lang.Class;:, > ??? ??? ?rru.funcArgs=[[January, February, March, April, May, June, July, August, September, > October, November, December, ]].getClass().toString()=class [Ljava.lang.Object; > > // // // RexxReflectJava9.processMethod(), bean=[CategoryAxis[id=xAxis, styleClass=axis]], > methodName=[SETCATEGORIES], coercedArgs=[[January, February, March, April, May, June, July, > August, September, October, November, December, ]] > > // // :( RexxReflectJava9.processMethod(), MethodType for Method [public final void > javafx.scene.chart.CategoryAxis.setCategories(javafx.collections.ObservableList)]: > "(ObservableList)void" > > // // :( RexxReflectJava9.processMethod(): INSTANCE, mh.bindTo("CategoryAxis[id=xAxis, > styleClass=axis]/class javafx.scene.chart.CategoryAxis").invokeWithArguments(...) > > ... add2cachedFieldsOrMethods(): rru.memberName=[SETCATEGORIES] -> > rru.keyMemberName=[SETCATEGORIES], rru.invocationType=[INVOKE], > cfm=[CachedFieldOrMethod[mhk=METHOD,mh=MethodHandle(CategoryAxis,ObservableList)void,parameterTypes=[interface > javafx.collections.ObservableList]]] > > I double checked that the only difference is in using java.lang.reflect.Method.invoke(...) which > makes the Throwable on the method handle invocation go away (and the monthnames to be shown by > Arrays.deepToString()). > > Here is the excerpt of the code section in question that allows the MethodHandle mh to be invoked > successfully with the same coerced argument: > > Class parameterTypes[] = tmpMethod.getParameterTypes(); > ??????????????? rru.result=tmpMethod.invoke(rru.bean,coercedArgs);????? // java.lang.reflect.Method > ... cut ... > ??????????? Class returnType=tmpMethod.getReturnType(); > ??????????? MethodType mt = MethodType.methodType(returnType, parameterTypes); > ... cut ... > ?????????? // access via MethodHandle with the rights of rru.firstExportedClz > ??????????????????????? mh=thisLookup.findVirtual(rru.firstExportedClz, methodName, mt); > ??????????????????????? // mh=thisLookup.unreflect(tmpMethod);? // same behaviour > > ??????????????? rru.result=mh.bindTo(rru.bean).invokeWithArguments(coercedArgs); // bind to > bean, invoke method > > Just commenting out the reflective invocation in line 2 above > ("rru.result=tmpMethod.invoke(rru.bean,coercedArgs); ") will cause the MethodHandle invocation to > fail with the reported Throwable! > > ---? > > Also it seems that java.util.Arrays.deepToString(...) is affected by this, if looking at the String > value it produces in both cases (different number of enclosing square brackets: the failing version > has another pair of enclosing square brackets). > > Here the debug code for creating the above outputs ("coercedArgs" will be returned by a coerce > method and will be null, if the "rru.funcArgs" arguments from Rexx could not be coerced according to > the "parameterTypes"): > > String tmpStrCoercedArgs= (coercedArgs==null ? null : > Arrays.deepToString(coercedArgs)+".getClass().toString()="+coercedArgs.getClass().toString() ); > > System.err.println("// // // RexxReflectJava9.processMethod(), coercedArgs="+tmpStrCoercedArgs > ?????????????????????????????????????????????????????????????? +",\n\t\t > parameterTypes="+Arrays.deepToString(parameterTypes)+".getClass().toString()="+parameterTypes.getClass().toString() > ?????????????????????????????????????????????????????????????? +":,\n\t\t > rru.funcArgs="+Arrays.deepToString(rru.funcArgs)+".getClass().toString()="+rru.funcArgs.getClass().toString() > ?????????????????????????????????????????????????????????????? ); > > --- > > This strange observation is on Windows 7: > > F:\work\svn\bsf4oorexx\trunk\bsf4oorexx.dev\source_cc>java -version > java version "9.0.1" > Java(TM) SE Runtime Environment (build 9.0.1+11) > Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode) > > --- > > Would anyone have an idea what the reason could be or have any advice? > > ---rony > > [1] Slides with the address book sample in question; look for the pictures in the section starting > at page: . > > P.S.: Yes, the debug output could be cleaner (evolved from many permutations and formatting in the > past weeks), however this is right from the battle-field and tidying everything up is next on the > todo list. > From Rony.Flatscher at wu.ac.at Tue Mar 13 12:16:49 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 13 Mar 2018 13:16:49 +0100 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> Message-ID: <22c7ba4e-4bc7-c223-a832-ac6fd9bbf7aa@wu.ac.at> Just to conclude this posting: first many thanks to Alan! Alan's little nutshell example about how to use MethodHandles.Lookup and employ MethodHandle s to invoke helped me a lot to get afloat with rewriting the reflection part for the bindings also using MH. The new reflection mechanism has been implemented such that on Java 1.6/6 and 1.7/7 it uses core reflection (on Java 1.7/7 MethodHandle only reflection runs in some errors that cannot be circumvented, hence using core reflection) reflection with the need to do setAccessible(true) in quite a few places. The support for Java 1.8/8 and 9 can use both, core reflection (without a need for setAccessible() at all!) and MethodHandle reflection (with one case found so far, that does not work on MH, but with core reflection). The type of reflection that is being carried out is in all my use cases (my bridge intentionally has been only supporting public classes and public members) not really specific/relevant to jigsaw. Just wanted to point that out explicitly. --- The only grief I have at the moment is with the meaning of "public": "public" has become a homonym in jigsaw, introducing a *lot* of confusion into Java (one time "public" is public, another time "public" is not public). This could be probably eased considerably IMHO, if there was a class Modifier introduced named "module" that would be set for reflective access to classes from non exported modules. This would have at least the following benefits: - communicating (and teaching!) about the publicness of a class can immediately be clarified ("is the class truly public or does it reside in a non-exported module?"), - also, programmatically it would be simple to learn whether a class object fetched reflectively is truly public or not by testing against the presence of such a "Module" modifier bit at runtime in Java 9 or higher. ---rony On 24.01.2018 20:48, Alan Bateman wrote: > On 24/01/2018 15:42, Rony G. Flatscher wrote: >> >> OK, now add to this the following class that uses p.C2 objects to access e.g. m() via it: >> >> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java >> >> public class UseC2 >> { >> ??? public static void main (String args[]) { >> ??????? p.C2 o=new p.C2(); >> ??????? System.out.println("o="+o); >> ??????? System.out.println("o.m()="+o.m()); >> ??? } >> } >> >> Compiling all three classes works. >> >> Running "UseC2" works and outputs: >> >> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 >> o=p.C2 at 66048bfd >> o.m()=-1 >> >> So it is possible to access m() via the p.C2 object from UseC2. > That's right and this is the reason for moving to the simpler example. > > To be absolutely sure then you should decompile C2.class to make sure that there isn't a bridge > method calling C1's m2(). If you change m() to be final then that will keep the bridge method from > complicating the picture. > > If you change UseC2 to use core reflection and you hit the issue because the Method object you get > is p.C1.m(). Attempting to invoke this will fail with IllegalAccessException. In your other mail > you show a code fragment where it catches exceptions and calls setAccessible - I'll guess that > this may have been masking the issue in the Rexx bridge. > > For completeness then you may want to try out the new reflection API. I realize you have to > compile to JDK 6 but I think you'll find it will work the same way as the invokevirtual that o.m() > compiles to. > > ??? MethodHandles.Lookup lookup = MethodHandles.lookup(); > ??? MethodType mt = MethodType.methodType(int.class); > ??? MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt); > ??? Object res = mh.invoke(o); > > -Alan From forax at univ-mlv.fr Tue Mar 13 13:17:15 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 13 Mar 2018 14:17:15 +0100 (CET) Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <22c7ba4e-4bc7-c223-a832-ac6fd9bbf7aa@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> <22c7ba4e-4bc7-c223-a832-ac6fd9bbf7aa@wu.ac.at> Message-ID: <1784743538.723568.1520947035585.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Rony G. Flatscher" > ?: "Alan Bateman" , "jigsaw-dev" > Envoy?: Mardi 13 Mars 2018 13:16:49 > Objet: Re: p example (Re: Reflection: how can one access public fields (members) in a superclass ? > Just to conclude this posting: first many thanks to Alan! > > Alan's little nutshell example about how to use MethodHandles.Lookup and employ > MethodHandle s to > invoke helped me a lot to get afloat with rewriting the reflection part for the > bindings also using > MH. The new reflection mechanism has been implemented such that on Java 1.6/6 > and 1.7/7 it uses core > reflection (on Java 1.7/7 MethodHandle only reflection runs in some errors that > cannot be > circumvented, hence using core reflection) reflection with the need to do > setAccessible(true) in > quite a few places. The support for Java 1.8/8 and 9 can use both, core > reflection (without a need > for setAccessible() at all!) and MethodHandle reflection (with one case found so > far, that does not > work on MH, but with core reflection). > > The type of reflection that is being carried out is in all my use cases (my > bridge intentionally has > been only supporting public classes and public members) not really > specific/relevant to jigsaw. Just > wanted to point that out explicitly. > > --- > > The only grief I have at the moment is with the meaning of "public": "public" > has become a homonym > in jigsaw, introducing a *lot* of confusion into Java (one time "public" is > public, another time > "public" is not public). > > This could be probably eased considerably IMHO, if there was a class Modifier > introduced named > "module" that would be set for reflective access to classes from non exported > modules. It already exists, but not at class level, you can open a module, or each package individually, in that case, setAccessible is allowed. > > This would have at least the following benefits: > > - communicating (and teaching!) about the publicness of a class can immediately > be clarified ("is > the class truly public or does it reside in a non-exported module?"), Teaching is not a big deal because you can teach package and module at the same time and say that it works like onions or russian dolls, you have module access, class access and member access, by example, a public field is not really public if the class is not public and the package not exported. If you take a look from the point of view of accessing a member (and not a class), things are regular at module level and class level. > > - also, programmatically it would be simple to learn whether a class object > fetched reflectively is > truly public or not by testing against the presence of such a "Module" modifier > bit at runtime in > Java 9 or higher. you can use Lookup.accessClass() [1]. > > ---rony R?mi [1] https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#accessClass-java.lang.Class- > > > On 24.01.2018 20:48, Alan Bateman wrote: >> On 24/01/2018 15:42, Rony G. Flatscher wrote: >>> >>> OK, now add to this the following class that uses p.C2 objects to access e.g. >>> m() via it: >>> >>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java >>> >>> public class UseC2 >>> { >>> ??? public static void main (String args[]) { >>> ??????? p.C2 o=new p.C2(); >>> ??????? System.out.println("o="+o); >>> ??????? System.out.println("o.m()="+o.m()); >>> ??? } >>> } >>> >>> Compiling all three classes works. >>> >>> Running "UseC2" works and outputs: >>> >>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 >>> o=p.C2 at 66048bfd >>> o.m()=-1 >>> >>> So it is possible to access m() via the p.C2 object from UseC2. >> That's right and this is the reason for moving to the simpler example. >> >> To be absolutely sure then you should decompile C2.class to make sure that there >> isn't a bridge >> method calling C1's m2(). If you change m() to be final then that will keep the >> bridge method from >> complicating the picture. >> >> If you change UseC2 to use core reflection and you hit the issue because the >> Method object you get >> is p.C1.m(). Attempting to invoke this will fail with IllegalAccessException. In >> your other mail >> you show a code fragment where it catches exceptions and calls setAccessible - >> I'll guess that >> this may have been masking the issue in the Rexx bridge. >> >> For completeness then you may want to try out the new reflection API. I realize >> you have to >> compile to JDK 6 but I think you'll find it will work the same way as the >> invokevirtual that o.m() >> compiles to. >> >> ??? MethodHandles.Lookup lookup = MethodHandles.lookup(); >> ??? MethodType mt = MethodType.methodType(int.class); >> ??? MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt); >> ??? Object res = mh.invoke(o); >> > > -Alan From greggwon at cox.net Tue Mar 13 16:32:22 2018 From: greggwon at cox.net (Gregg Wonderly) Date: Tue, 13 Mar 2018 11:32:22 -0500 Subject: why is modularity changing the rules of inheritance? In-Reply-To: References: Message-ID: This is pretty much what was lamented about on the list as the project proceeded. In the end, the restrictions you see are because all the system classes in the Java VM implementation need to be protected in this way to provide the opportunity for evolution of the implementations underlying the JVM. Rather than implementing some customization of visibility ?into? specific system classes, we all get to deal with having visibility changes that break existing code which uses introspection for IoC etc. Practically, Jigsaw feels like too little too late for Java improvement over all. It?s way past the time when people would be satisfied to see Java fixed in places where underlying APIs were not sufficient and so introspection based fixes ended up being made, or base JVM classes replaced with bootclasspath. There seems to be an honest effort to try and fix things in the base of Java by this work, but the cloud environment is being overrun by .Net, just because of the number of developers being trained there, vs the J2EE disaster. .Net is not an attractive solution for stability reasons however. Additionally, the desktop environment is so overrun by other platforms due to the lack of focus on media support and fixes to Swing/AWT problems, that Java on the desktop is probably not as interesting as it once was. Swift, Go, .Net and a few other platforms have much more productivity for creating usable applications, and are being taught and used much more prolifically than Java. Apple pulled away from Java completely as Swift came flying into view. That was a pretty good number of nails into the coffin. Oracle has a bad name in many places because of ?Android?, ?Jenkins?, ?MySQL? and other Oracle things around open source arguments. People are standing at a distance from anything that Oracle is doing because they tire of the argumentative and restrictive control that is constantly changing for the worse. It?s really a sad state of affairs given the opportunity for unification of programming language/platform that Java once offered? Gregg > On Mar 13, 2018, at 4:23 AM, Bernard Amade wrote: > > Hello all (OOps! wrote "Hell all" ) > > just curious to read about the rationale behind this behaviour: > if you have a code in moduleB that inherits from code in moduleA, modularity wise the inherited methods behave as if they were invoked from moduleA (not from moduleB). see an example at the end of this post. > Na?vely found this behaviour *very bad* but I suppose that if it were not the case then the situation could be worse... so I would like to read the rationale behind this. > > thanks > > code of a service definition in moduleA: > ----------------------------------------------- > public abstract class PropertiesProvider { > public Properties get(String domain) { > Class clazz =this.getClass(); > System.out.println(" CLASS " +clazz); > try { > Object obj = clazz.getConstructor().newInstance(); > System.out.println("Object " + obj); > > } catch (Exception exc) { > exc.printStackTrace(); > } > InputStream is = clazz.getResourceAsStream(domain) ; > if (is != null) { > Properties props = new Properties(); > try { > props.load(is); > return props; > } catch (IOException e) { > System.getLogger("").log(System.Logger.Level.WARNING,e) ; > } > > } > and so on ... > now if in module B you just create a Service by inheriting from this class: > - the className is correct (an instance of the class has been created for the service) > - you can't invoke newInstance from that very class!(if the class is not opened or exported) > - you can't get the Resource (or other files) unless they are explicilty opened. > if you copy this code in the class implementing the service everything just works fine > so there is a difference between the same code being explicitly in the class or inherited! > (got a headache trying to find something to say for people learning Java) From Rony.Flatscher at wu.ac.at Wed Mar 14 14:20:40 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 14 Mar 2018 15:20:40 +0100 Subject: ad 'module' modifier idea (Re: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <1784743538.723568.1520947035585.JavaMail.zimbra@u-pem.fr> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> <22c7ba4e-4bc7-c223-a832-ac6fd9bbf7aa@wu.ac.at> <1784743538.723568.1520947035585.JavaMail.zimbra@u-pem.fr> Message-ID: <32fe78a2-d72b-c088-dc16-98efc2fb9b04@wu.ac.at> Just changed the subject to indicate better its content. On 13.03.2018 14:17, Remi Forax wrote: ... cut ... > --- > > The only grief I have at the moment is with the meaning of "public": "public" > has become a homonym > in jigsaw, introducing a *lot* of confusion into Java (one time "public" is > public, another time > "public" is not public). > > This could be probably eased considerably IMHO, if there was a class Modifier > introduced named > "module" that would be set for reflective access to classes from non exported > modules. > It already exists, but not at class level, yes, something like (e.g. from ): if (Modifier.isPublic(clazz.getModifiers()) && clazz.getModule().isExported(clazz.getPackageName(), caller.getModule()) ... > you can open a module, or each package individually, in that case, setAccessible is allowed. yes, however this adds the need to learn the new concept "open" (which is fine for advanced Java programmers) >> This would have at least the following benefits: >> >> - communicating (and teaching!) about the publicness of a class can immediately >> be clarified ("is >> the class truly public or does it reside in a non-exported module?"), > Teaching is not a big deal because you can teach package and module at the same time and say that it works like onions or russian dolls, nice metapher! > you have module access, class access and member access, by example, a public field is not really public if the class is not public and the package not exported. > If you take a look from the point of view of accessing a member (and not a class), things are regular at module level and class level. in my case, if thinking of teaching it is thinking of? "beginner"/"end-user"/"casual" programmer type of people (business administration students who are interested in learning programming, however not much time available in the curriculum to teach it) > >> - also, programmatically it would be simple to learn whether a class object >> fetched reflectively is >> truly public or not by testing against the presence of such a "Module" modifier >> bit at runtime in >> Java 9 or higher. > you can use Lookup.accessClass() [1]. yes, but this a quite advanced concept, it takes quite some time before arriving and being able to take advantage of it > [1] https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#accessClass-java.lang.Class- > --- There are at least two aspects in the context of the "public" homonym: * one is the aspect of conceptually understanding it ("why are 'public' classes sometimes not 'public'?"), and * one is to be able to personally test it (maximum learning effect) with minimal Java skills, which is why I would prefer the info that a class is from a non-exported module be given with a? hypothetical "module" identfier with the class object (if absent, the class is indeed public to the world). This way, if a student (or any - expert or casual - Java programmer) uses "o.getClass().toString()" would see e.g. "module class xyz.SomeClass" and immediately know that this class is from a non-exported module. This would simplify this issue considerably IMHO. ---rony From paul.sandoz at oracle.com Thu Mar 15 17:06:41 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 15 Mar 2018 10:06:41 -0700 Subject: RFR 8193033 remove terminally deprecated sun.misc.Unsafe.defineClass Message-ID: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> Hi, Please review this patch to remove sun.misc.Unsafe.defineClass in 11. There has been much outreach, by Alan and the Jigsaw team, about its public replacement MethodHandles.Lookup.defineClass. CSR is here: https://bugs.openjdk.java.net/browse/JDK-8199699 Thanks, Paul. diff -r 3c0a12972165 src/jdk.unsupported/share/classes/sun/misc/Unsafe.java --- a/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 08:11:01 2018 -0700 +++ b/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 09:51:00 2018 -0700 @@ -811,25 +811,6 @@ /// random trusted operations from JNI: /** - * Tells the VM to define a class, without security checks. By default, the - * class loader and protection domain come from the caller's class. - * - * @deprecated Use {@link java.lang.invoke.MethodHandles.Lookup#defineClass MethodHandles.Lookup#defineClass} - * to define a class to the same class loader and in the same runtime package - * and {@linkplain java.security.ProtectionDomain protection domain} of a - * given {@code Lookup}'s {@linkplain java.lang.invoke.MethodHandles.Lookup#lookupClass() lookup class}. - * - * @see java.lang.invoke.MethodHandles.Lookup#defineClass(byte[]) - */ - @Deprecated(since="9", forRemoval=true) - @ForceInline - public Class defineClass(String name, byte[] b, int off, int len, - ClassLoader loader, - ProtectionDomain protectionDomain) { - return theInternalUnsafe.defineClass(name, b, off, len, loader, protectionDomain); - } - - /** * Defines a class but does not make it known to the class loader or system dictionary. *

* For each CP entry, the corresponding CP patch must either be null or have From chris.hegarty at oracle.com Thu Mar 15 17:16:47 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 15 Mar 2018 17:16:47 +0000 Subject: RFR 8193033 remove terminally deprecated sun.misc.Unsafe.defineClass In-Reply-To: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> References: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> Message-ID: <24D7FB4D-291D-41FB-BF10-F52A5E45CD09@oracle.com> > On 15 Mar 2018, at 17:06, Paul Sandoz wrote: > > Hi, > > Please review this patch to remove sun.misc.Unsafe.defineClass in 11. > > There has been much outreach, by Alan and the Jigsaw team, about its public replacement MethodHandles.Lookup.defineClass. > > CSR is here: > > https://bugs.openjdk.java.net/browse/JDK-8199699 Looks good Paul. I don?t think that a CSR is strictly needed, but does no harm. -Chris. > Thanks, > Paul. > > > diff -r 3c0a12972165 src/jdk.unsupported/share/classes/sun/misc/Unsafe.java > --- a/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 08:11:01 2018 -0700 > +++ b/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 09:51:00 2018 -0700 > @@ -811,25 +811,6 @@ > /// random trusted operations from JNI: > > /** > - * Tells the VM to define a class, without security checks. By default, the > - * class loader and protection domain come from the caller's class. > - * > - * @deprecated Use {@link java.lang.invoke.MethodHandles.Lookup#defineClass MethodHandles.Lookup#defineClass} > - * to define a class to the same class loader and in the same runtime package > - * and {@linkplain java.security.ProtectionDomain protection domain} of a > - * given {@code Lookup}'s {@linkplain java.lang.invoke.MethodHandles.Lookup#lookupClass() lookup class}. > - * > - * @see java.lang.invoke.MethodHandles.Lookup#defineClass(byte[]) > - */ > - @Deprecated(since="9", forRemoval=true) > - @ForceInline > - public Class defineClass(String name, byte[] b, int off, int len, > - ClassLoader loader, > - ProtectionDomain protectionDomain) { > - return theInternalUnsafe.defineClass(name, b, off, len, loader, protectionDomain); > - } > - > - /** > * Defines a class but does not make it known to the class loader or system dictionary. > *

> * For each CP entry, the corresponding CP patch must either be null or have From mandy.chung at oracle.com Thu Mar 15 17:36:00 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 15 Mar 2018 10:36:00 -0700 Subject: RFR 8193033 remove terminally deprecated sun.misc.Unsafe.defineClass In-Reply-To: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> References: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> Message-ID: +1 Mandy On 3/15/18 10:06 AM, Paul Sandoz wrote: > Hi, > > Please review this patch to remove sun.misc.Unsafe.defineClass in 11. > > There has been much outreach, by Alan and the Jigsaw team, about its public replacement MethodHandles.Lookup.defineClass. > > CSR is here: > > https://bugs.openjdk.java.net/browse/JDK-8199699 > > Thanks, > Paul. > > > diff -r 3c0a12972165 src/jdk.unsupported/share/classes/sun/misc/Unsafe.java > --- a/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 08:11:01 2018 -0700 > +++ b/src/jdk.unsupported/share/classes/sun/misc/Unsafe.java Thu Mar 15 09:51:00 2018 -0700 > @@ -811,25 +811,6 @@ > /// random trusted operations from JNI: > > /** > - * Tells the VM to define a class, without security checks. By default, the > - * class loader and protection domain come from the caller's class. > - * > - * @deprecated Use {@link java.lang.invoke.MethodHandles.Lookup#defineClass MethodHandles.Lookup#defineClass} > - * to define a class to the same class loader and in the same runtime package > - * and {@linkplain java.security.ProtectionDomain protection domain} of a > - * given {@code Lookup}'s {@linkplain java.lang.invoke.MethodHandles.Lookup#lookupClass() lookup class}. > - * > - * @see java.lang.invoke.MethodHandles.Lookup#defineClass(byte[]) > - */ > - @Deprecated(since="9", forRemoval=true) > - @ForceInline > - public Class defineClass(String name, byte[] b, int off, int len, > - ClassLoader loader, > - ProtectionDomain protectionDomain) { > - return theInternalUnsafe.defineClass(name, b, off, len, loader, protectionDomain); > - } > - > - /** > * Defines a class but does not make it known to the class loader or system dictionary. > *

> * For each CP entry, the corresponding CP patch must either be null or have From Alan.Bateman at oracle.com Thu Mar 15 17:39:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 15 Mar 2018 17:39:50 +0000 Subject: RFR 8193033 remove terminally deprecated sun.misc.Unsafe.defineClass In-Reply-To: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> References: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> Message-ID: <2224e377-de1e-2dfb-f70c-026ba8eac4ce@oracle.com> On 15/03/2018 17:06, Paul Sandoz wrote: > Hi, > > Please review this patch to remove sun.misc.Unsafe.defineClass in 11. > > There has been much outreach, by Alan and the Jigsaw team, about its public replacement MethodHandles.Lookup.defineClass. > > CSR is here: > > https://bugs.openjdk.java.net/browse/JDK-8199699 > Looks good, I assume the import of java.security.ProtectionDomain can be removed too. -Alan From paul.sandoz at oracle.com Thu Mar 15 18:06:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 15 Mar 2018 11:06:17 -0700 Subject: RFR 8193033 remove terminally deprecated sun.misc.Unsafe.defineClass In-Reply-To: <2224e377-de1e-2dfb-f70c-026ba8eac4ce@oracle.com> References: <5E842D0A-B7F3-44BF-94EA-2CD7B8DA9057@oracle.com> <2224e377-de1e-2dfb-f70c-026ba8eac4ce@oracle.com> Message-ID: > On Mar 15, 2018, at 10:39 AM, Alan Bateman wrote: > > On 15/03/2018 17:06, Paul Sandoz wrote: >> Hi, >> >> Please review this patch to remove sun.misc.Unsafe.defineClass in 11. >> >> There has been much outreach, by Alan and the Jigsaw team, about its public replacement MethodHandles.Lookup.defineClass. >> >> CSR is here: >> >> https://bugs.openjdk.java.net/browse/JDK-8199699 >> > Looks good, I assume the import of java.security.ProtectionDomain can be removed too. > Yes, thanks, Paul. From martijnverburg at gmail.com Fri Mar 23 10:19:51 2018 From: martijnverburg at gmail.com (Martijn Verburg) Date: Fri, 23 Mar 2018 10:19:51 +0000 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) Message-ID: Hi All, Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j pms-negative-benefits.html He does raise some interesting points. Perhaps it's worth spending some time before 11 is out to see if there is a combination of Maven and JPMS changes / clarifications that can address some of these pain points. If Stephen is missing something, then that's an area we could document (and/or publicise better if already documented. That is something I and others can help to address through the usual JUG Leaders / Champions / Info Q etc channels. @Stephen are you able to share the source code? It may be that there is another path that could be explored to mitigate some of the challenges you faced. @Robert - I've been thinking about the difficulty of getting enough engineering time for Maven committers to add support for major features like JPMS. I'll start a separate thread with you on that. Cheers, Martijn From david.lloyd at redhat.com Fri Mar 23 11:43:21 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 23 Mar 2018 06:43:21 -0500 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: On Fri, Mar 23, 2018 at 5:19 AM, Martijn Verburg wrote: > Hi All, > > Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j > pms-negative-benefits.html > > He does raise some interesting points. Perhaps it's worth spending some > time before 11 is out to see if there is a combination of Maven and JPMS > changes / clarifications that can address some of these pain points. > > If Stephen is missing something, then that's an area we could document > (and/or publicise better if already documented. That is something I and > others can help to address through the usual JUG Leaders / Champions / Info > Q etc channels. > > @Stephen are you able to share the source code? It may be that there is > another > path that could be explored to mitigate some of the challenges you faced. > > @Robert - I've been thinking about the difficulty of getting enough > engineering time for Maven committers to add support for major features > like JPMS. I'll start a separate thread with you on that. Case in point: I've had https://github.com/apache/maven-compiler-plugin/pull/1 sitting out there since January. This commit is a key to what is at least a somewhat workable strategy for MR JARs and multi-JVM build and testing for us. Robert has said that the change is acceptable, but indicated that there may be a resource problem in terms of getting the plugin to a releasable state. I (and others I work with) are more than willing to contribute what extra time we have to fixing or improving things in Maven and its plugins, and even in the JDK itself, but the other side of that is that the fixes do need to be accepted in a timely manner. With Java 11 just around the corner, the time is now. -- - DML From scolebourne at joda.org Fri Mar 23 12:51:06 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 23 Mar 2018 12:51:06 +0000 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: On 23 March 2018 at 10:19, Martijn Verburg wrote: > Stephen's comments are in his blog post: > http://blog.joda.org/2018/03/jpms-negative-benefits.html Firstly, I want to emphasise that my goal is effectively that of a retrospective, to examine what hasn't worked so well and to improve things from where we are. The jigsaw team did a great job in getting the feature out at all - I want to make sure it is used rather than ignored or used as a blocker to progressing the JDK. > @Stephen are you able to share the source code? The source code is all public: https://github.com/ThreeTen/threeten-extra https://github.com/JodaOrg/joda-parent https://github.com/JodaOrg/joda-convert https://github.com/JodaOrg/joda-beans While some of the problems are tool-based, the most fundamental issues are about JPMS itself. At the heart of the problem is the split between class-path and module-path. Since this split has happened now, and can't "unhappen" what is needed is a way to make it easier to manage (fix the problems the split has created). As it stands, a library developer cannot control whether they run on the class-path or module-path. This increases the bug-surface of the library, and requires testing in two different environments (which is not widely known). Without being able to insist that a library is on the module-path it is also clear that the benefits of strong encapsulation and reliable configuration don't apply to library consumers. My belief is that a way needs to be found for a library author to insist that their library is run as a named module. There are probably a number of ways this could be achieved - I'm interested in whether the change makes things better, not what the specific change is. One approach is to say that modular jar files are always treated as named modules: - a modular jar on the class-path is treated as being named, not part of the unnamed module It is therefore encapsulated, but depends on the unnamed module (class-path), so does not have full reliable configuration. There is the potential for some incompatibility with this change where code that uses the modular jar now can't access the encapsulated packages, but this is a Good thing (as the library author specifically coded for that encapsulation). Any incompatibilities are smaller than JPMS has already caused, and could be managed with the existing command line flags. I would hope this does not require a JVM spec change to achieve. A second approach is to say that the module author must mark modules strict: - a module author could mark a module as "strict" in module-info.java so that it is not permitted to be on the class-path. This does not force the whole application to move to the module-path - only strict modules and their dependencies would need to move. At runtime, if a strict module is found on the class-path an error occurs. I suspect this requires a JVM spec change, so may be harder. There may well be more approaches. My gut feeling is that the first approach is the better one, being relatively simple and implementable, but of course I may be wrong. Ultimately, the requirement is that library authors who go to the effort of adding module-info.java should see some benefits in doing so, where today they only have increased cost through running as both an unnamed and named module. thanks Stephen From forax at univ-mlv.fr Fri Mar 23 13:08:41 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 23 Mar 2018 14:08:41 +0100 (CET) Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: <120535893.2533219.1521810521477.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stephen Colebourne" > ?: "jigsaw-dev" > Envoy?: Vendredi 23 Mars 2018 13:51:06 > Objet: Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) > On 23 March 2018 at 10:19, Martijn Verburg wrote: >> Stephen's comments are in his blog post: >> http://blog.joda.org/2018/03/jpms-negative-benefits.html > > Firstly, I want to emphasise that my goal is effectively that of a > retrospective, to examine what hasn't worked so well and to improve > things from where we are. The jigsaw team did a great job in getting > the feature out at all - I want to make sure it is used rather than > ignored or used as a blocker to progressing the JDK. > > >> @Stephen are you able to share the source code? > > The source code is all public: > https://github.com/ThreeTen/threeten-extra > https://github.com/JodaOrg/joda-parent > https://github.com/JodaOrg/joda-convert > https://github.com/JodaOrg/joda-beans > > While some of the problems are tool-based, the most fundamental issues > are about JPMS itself. > > At the heart of the problem is the split between class-path and > module-path. Since this split has happened now, and can't "unhappen" > what is needed is a way to make it easier to manage (fix the problems > the split has created). > > As it stands, a library developer cannot control whether they run on > the class-path or module-path. This increases the bug-surface of the > library, and requires testing in two different environments (which is > not widely known). Without being able to insist that a library is on > the module-path it is also clear that the benefits of strong > encapsulation and reliable configuration don't apply to library > consumers. > > My belief is that a way needs to be found for a library author to > insist that their library is run as a named module. There are probably > a number of ways this could be achieved - I'm interested in whether > the change makes things better, not what the specific change is. > > One approach is to say that modular jar files are always treated as > named modules: > > - a modular jar on the class-path is treated as being named, not part > of the unnamed module It is therefore encapsulated, but depends on the > unnamed module (class-path), so does not have full reliable > configuration. There is the potential for some incompatibility with > this change where code that uses the modular jar now can't access the > encapsulated packages, but this is a Good thing (as the library author > specifically coded for that encapsulation). Any incompatibilities are > smaller than JPMS has already caused, and could be managed with the > existing command line flags. I would hope this does not require a JVM > spec change to achieve. > > A second approach is to say that the module author must mark modules strict: > > - a module author could mark a module as "strict" in module-info.java > so that it is not permitted to be on the class-path. This does not > force the whole application to move to the module-path - only strict > modules and their dependencies would need to move. At runtime, if a > strict module is found on the class-path an error occurs. I suspect > this requires a JVM spec change, so may be harder. > > There may well be more approaches. My gut feeling is that the first > approach is the better one, being relatively simple and implementable, > but of course I may be wrong. > > Ultimately, the requirement is that library authors who go to the > effort of adding module-info.java should see some benefits in doing > so, where today they only have increased cost through running as both > an unnamed and named module. > > thanks > Stephen You can check dynamically if a class is in the unamed module (classpath) or not and emit an error message. This is exactly what the VM does when the class version is wrong. R?mi From david.lloyd at redhat.com Fri Mar 23 13:17:23 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 23 Mar 2018 08:17:23 -0500 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne wrote: > One approach is to say that modular jar files are always treated as > named modules: > > - a modular jar on the class-path is treated as being named, not part > of the unnamed module It is therefore encapsulated, but depends on the > unnamed module (class-path), so does not have full reliable > configuration. There is the potential for some incompatibility with > this change where code that uses the modular jar now can't access the > encapsulated packages, but this is a Good thing (as the library author > specifically coded for that encapsulation). Any incompatibilities are > smaller than JPMS has already caused, and could be managed with the > existing command line flags. I would hope this does not require a JVM > spec change to achieve. This would cause problems for containers which do not use JPMS yet (which is to say: containers); such frameworks may behave poorly (having an expectation that they were loaded as JPMS modules) or fail to load (if there is some kind of hypothetical enforcement at a JVM level). > A second approach is to say that the module author must mark modules strict: > > - a module author could mark a module as "strict" in module-info.java > so that it is not permitted to be on the class-path. This does not > force the whole application to move to the module-path - only strict > modules and their dependencies would need to move. At runtime, if a > strict module is found on the class-path an error occurs. I suspect > this requires a JVM spec change, so may be harder. I don't think this would resolve the container problem, though it would allow containers to refuse to load such JARs. I'm not sure that's a net improvement for users though. > Ultimately, the requirement is that library authors who go to the > effort of adding module-info.java should see some benefits in doing > so, where today they only have increased cost through running as both > an unnamed and named module. I agree, we've effectively doubled the testing effort for frameworks due to JPMS, and tripled in the case where frameworks must include MR JAR supplements for 8 vs 9+. In a word, this sucks. -- - DML From jason.greene at redhat.com Fri Mar 23 13:25:20 2018 From: jason.greene at redhat.com (Jason Greene) Date: Fri, 23 Mar 2018 08:25:20 -0500 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: > On Mar 23, 2018, at 7:51 AM, Stephen Colebourne wrote: > > Without being able to insist that a library is on > the module-path it is also clear that the benefits of strong > encapsulation and reliable configuration don't apply to library > consumers. Why would a framework author ever want to cut off 99% of their users that are not using JPMS, most of which are completely happy with the arguably more flexible classpath? I think the only way you solve this problem is if you resurrect (and extend) some of the original goals to support common unification and bridging across runtime models. Then the industry as a whole could be pushing adoption. -Jason From scolebourne at joda.org Fri Mar 23 13:49:23 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 23 Mar 2018 13:49:23 +0000 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: On 23 March 2018 at 13:17, David Lloyd wrote: > On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne > wrote: >> One approach is to say that modular jar files are always treated as >> named modules: >> >> - a modular jar on the class-path is treated as being named, not part >> of the unnamed module It is therefore encapsulated, but depends on the >> unnamed module (class-path), so does not have full reliable >> configuration. There is the potential for some incompatibility with >> this change where code that uses the modular jar now can't access the >> encapsulated packages, but this is a Good thing (as the library author >> specifically coded for that encapsulation). Any incompatibilities are >> smaller than JPMS has already caused, and could be managed with the >> existing command line flags. I would hope this does not require a JVM >> spec change to achieve. > > This would cause problems for containers which do not use JPMS yet > (which is to say: containers); such frameworks may behave poorly > (having an expectation that they were loaded as JPMS modules) or fail > to load (if there is some kind of hypothetical enforcement at a JVM > level). The classes in the modular jar file would still be in the same classloader (because same classloader module loading is the standard strategy). Other than encapsulating internal packages, the container really shouldn't see any changes should it? At some point containers will support JPMS anyway. Stephen From david.lloyd at redhat.com Fri Mar 23 14:00:03 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 23 Mar 2018 09:00:03 -0500 Subject: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne) In-Reply-To: References: Message-ID: On Fri, Mar 23, 2018 at 8:49 AM, Stephen Colebourne wrote: > On 23 March 2018 at 13:17, David Lloyd wrote: >> On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne >> wrote: >>> One approach is to say that modular jar files are always treated as >>> named modules: >>> >>> - a modular jar on the class-path is treated as being named, not part >>> of the unnamed module It is therefore encapsulated, but depends on the >>> unnamed module (class-path), so does not have full reliable >>> configuration. There is the potential for some incompatibility with >>> this change where code that uses the modular jar now can't access the >>> encapsulated packages, but this is a Good thing (as the library author >>> specifically coded for that encapsulation). Any incompatibilities are >>> smaller than JPMS has already caused, and could be managed with the >>> existing command line flags. I would hope this does not require a JVM >>> spec change to achieve. >> >> This would cause problems for containers which do not use JPMS yet >> (which is to say: containers); such frameworks may behave poorly >> (having an expectation that they were loaded as JPMS modules) or fail >> to load (if there is some kind of hypothetical enforcement at a JVM >> level). > > The classes in the modular jar file would still be in the same > classloader (because same classloader module loading is the standard > strategy). Other than encapsulating internal packages, the container > really shouldn't see any changes should it? At some point containers > will support JPMS anyway. The container won't see any changes, but the framework will if the container is not creating a JPMS module for it. At some point, containers _may_ support JPMS. However, this seems to be turning into quite a technical challenge in practice. And until there are some implementations supporting JPMS, there will be no specs for supporting JPMS (at least, unless the world has gone mad, which perhaps it has). -- - DML From nipa at codefx.org Sun Mar 25 07:32:33 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sun, 25 Mar 2018 09:32:33 +0200 Subject: Accessing internals in Java 11 Message-ID: <32ed1dce-7ef2-83a3-101f-5b1dba1b440f@codefx.org> Hi! On Java 9 and 10, the JPMS is forgiving when it comes to illegal access of JDK internals and jdk.unsupported offers classes like Unsafe or Signal. The same is true for 11-b5. Are there any plans to change this, i.e. will Java 11 become stricter before the release? Would --illegal-access get a different default value or might jdk.unsupported get smaller? so long ... Nicolai -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From Alan.Bateman at oracle.com Sun Mar 25 13:57:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 25 Mar 2018 14:57:38 +0100 Subject: Accessing internals in Java 11 In-Reply-To: <32ed1dce-7ef2-83a3-101f-5b1dba1b440f@codefx.org> References: <32ed1dce-7ef2-83a3-101f-5b1dba1b440f@codefx.org> Message-ID: On 25/03/2018 08:32, Nicolai Parlog wrote: > Hi! > > On Java 9 and 10, the JPMS is forgiving when it comes to illegal > access of JDK internals and jdk.unsupported offers classes like Unsafe > or Signal. The same is true for 11-b5. > > Are there any plans to change this, i.e. will Java 11 become stricter > before the release? Would --illegal-access get a different default > value or might jdk.unsupported get smaller? > TBD on dialing up the encapsulation of JDK internals, JDK 11 might be too soon. I'm sure there will be lots of discussion on this once the time comes. The only changes to class in the jdk.unsupported module so far have been the removal of Reflection.getClassClass and Unsafe.defineClass. There were terminally deprecated (@Deprecated forRemoval=true) so shouldn't be a surprise to anyone. -Alan From mark.reinhold at oracle.com Mon Mar 26 18:08:38 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 26 Mar 2018 11:08:38 -0700 (PDT) Subject: The baby and the bathwater Message-ID: <20180326180838.7135C196C48@eggemoggin.niobe.net> Stephen Colebourne's recent blog entry [1] contains many true statements, along with some reasonable advice for library maintainers. To summarize: - As of Java 9, with Jigsaw, there are two ways in which a library can be used: Either on the traditional class path, or on the newfangled module path. If you maintain a library but don't modularize it then it can still -- unbeknownst to you -- be used on the module path as an automatic module. - When code runs on the module path there are some differences in the behavior of some pre-9 APIs, in particular those related to resource lookup. - As a consequence, if you maintain a library that's intended to work on Java 9 or later then you should test it on both the class path and the module path, even if you do nothing to convert your library to a module. If your library doesn't work on the module path then you should either fix it or document that limitation. - If you don't modularize your library, or at least claim an automatic module name for it via the `Automatic-Module-Name` manifest entry, then you potentially block the maintainers of libraries that depend upon yours from modularizing their own libraries. - The tools that we use, and in particular Maven, could be improved. It's difficult to compile the classes for a modular JAR file that's intended to work on the class path of pre-9 releases, it's difficult to test a library on both the class path and the module path, and various Maven plugins still need to be upgraded to handle (or else ignore) `module-info.java` files. (Stephen has helpfully filed issues in the appropriate bug trackers for some of these problems.) - Some old tools, bytecode libraries, and other systems fail when they encounter `module-info.class` files or multi-release JAR files. >From these points Stephen concludes that the module system, "as currently designed, has 'negative benefits' for open source libraries," saying that this is primarily because "the split (bifurcation) of the module-path from the class-path is an absolute nightmare." Hyperbole aside, Stephen's main complaint here is only about the need to test a library on both the class path and the module path if it's intended to work on Java 9 or later. With automated testing this shouldn't, in principle, be a huge burden, but still it's worth asking the question: Could we have avoided the need for such dual testing if we hadn't introduced the module path as separate from the class path? Consider, as a thought experiment, an alternative Jigsaw design that didn't have a separate module path, and instead treated modular JARs on the class path as modules rather than traditional JAR files. You wouldn't have to dual-test if your baseline is Java 9 or later, but if you want to support earlier releases with the same artifact then you'd still have to test on the class path. With the actual Jigsaw design you do need to dual-test your library when your baseline is Java 9 or later. There is, however, a benefit to this: If someone uses your library in an application that works on the Java 8 class path today then they can migrate it to the Java 9 (or later) class path and then, when they're ready, move your library (and perhaps some others) over to the module path. (There were many other reasons to define the module path as separate from the class path, but those aren't directly relevant here.) The tradeoff, then, is a little bit more dual testing on the part of library maintainers in exchange for greater flexibility for those who will migrate existing applications to Java 9 or later releases. Many library maintainers will be reluctant to baseline to Java 9 (or later) for a while yet, so they'll be dual-testing anyway, so I think this was the right tradeoff. Stephen closes with a specific suggestion: "There needs to be a way for a library author to insist that the modular jar file they are producing can only be run on the module-path (with any attempt to use it on the class-path preventing application startup). This would eliminate the need for testing both class-path and module-path." Yes, this would eliminate the need for dual testing, but only if you're willing to baseline to Java 11. As with a unified class/module path, however, if you want your library to work on earlier releases then you'd also, still, have to test on the class path, and you'd make it harder for application maintainers to migrate old class-path applications. I don't think this idea is worth pursuing. What ideas are worth pursuing? We should, by all means, continue to improve our tools, Jigsaw itself, and the rest of the JDK. Several of us here collaborated on the initial support for modular development in Maven, but clearly there's more to do. If nothing else, the Surefire plugin should be able to test a library on both the class path and the module path. There's also at least one improvement in the JDK worth considering, which a few of us have already discussed, namely a small enhancement to javac that would allow a single invocation to compile a module, in module mode [2], but target class files other than `module-info.class` to an earlier release [3]. To sum up: We have more work to do, based on practical experience. This shouldn't be a surprise, with a change to the platform of this scope. Let's keep doing that work, let's not unduly alarm ourselves or anyone else, and please let's not throw out the baby with the bathwater. - Mark [1] http://blog.joda.org/2018/03/jpms-negative-benefits.html [2] http://openjdk.java.net/jeps/261#Compile-time [3] https://bugs.openjdk.java.net/browse/JDK-8200254 From scolebourne at joda.org Mon Mar 26 21:57:24 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 26 Mar 2018 22:57:24 +0100 Subject: The baby and the bathwater In-Reply-To: <20180326180838.7135C196C48@eggemoggin.niobe.net> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> Message-ID: On 26 March 2018 at 19:08, wrote: > Stephen Colebourne's recent blog entry Thanks for the thoughtful reply, of which I agree with much of it. > Stephen's main complaint here is only about the need > to test a library on both the class path and the module path if it's > intended to work on Java 9 or later. With automated testing this > shouldn't, in principle, be a huge burden, To a degree this depends on the size of your test suite. Some suites are large, and running the entire suite twice in continuous integration could be onerous. I think the main complaint however is more subtle than a need to test twice. It is the need to test twice _forevermore_. ie. if this were just a transition phase, which would pass when Java 11 is the baseline, the situation would be painful but manageable. But as things stand, there is no future time when my module will be guaranteed to be treated as a module. > Stephen closes with a specific suggestion: > [snip] > Yes, this would eliminate the need for dual testing, but only if you're > willing to baseline to Java 11. And that is exactly the point. At some point, Java 11 will be the new baseline. The trade-offs jigsaw chose for the Java 8 to 9 transition will at some point not be the right ones for the long term. There has to be a time when a library developer can rely on strong encapsulation and when the class-path can't just be used to completely nullify module-info.java. Otherwise, whats the point in modularisation? I'm arguing that it should be the module authors choice to apply the tougher rules, but I'm very happy to hear other alternatives in the problem-space. > There's also at least one improvement in the JDK worth > considering, which a few of us have already discussed, namely a small > enhancement to javac that would allow a single invocation to compile a > module, in module mode [2], but target class files other than > `module-info.class` to an earlier release [3]. +1 Stephen From cedric.champeau at gmail.com Tue Mar 27 07:15:34 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Tue, 27 Mar 2018 09:15:34 +0200 Subject: The baby and the bathwater In-Reply-To: References: <20180326180838.7135C196C48@eggemoggin.niobe.net> Message-ID: Dual testing is a minimum. In practice, it depends on the kind of tests. Typically, before JDK 9 for unit tests you never needed a jar to execute unit tests. Maven happens to built it, but in practice a class directory + resources is enough (what Gradle does when it knows a jar is not required). For integration or functional tests, you need the jar though, which means there are more combinations to test (class directory, jar on classpath, jar on module path, different runtime, ...). This is not necessarily a problem, we have the tools to do this, but the setup might not be super convenient. 2018-03-26 23:57 GMT+02:00 Stephen Colebourne : > On 26 March 2018 at 19:08, wrote: > > Stephen Colebourne's recent blog entry > > Thanks for the thoughtful reply, of which I agree with much of it. > > > Stephen's main complaint here is only about the need > > to test a library on both the class path and the module path if it's > > intended to work on Java 9 or later. With automated testing this > > shouldn't, in principle, be a huge burden, > > To a degree this depends on the size of your test suite. Some suites > are large, and running the entire suite twice in continuous > integration could be onerous. > > I think the main complaint however is more subtle than a need to test > twice. It is the need to test twice _forevermore_. > > ie. if this were just a transition phase, which would pass when Java > 11 is the baseline, the situation would be painful but manageable. But > as things stand, there is no future time when my module will be > guaranteed to be treated as a module. > > > Stephen closes with a specific suggestion: > > [snip] > > Yes, this would eliminate the need for dual testing, but only if you're > > willing to baseline to Java 11. > > And that is exactly the point. At some point, Java 11 will be the new > baseline. The trade-offs jigsaw chose for the Java 8 to 9 transition > will at some point not be the right ones for the long term. There has > to be a time when a library developer can rely on strong encapsulation > and when the class-path can't just be used to completely nullify > module-info.java. Otherwise, whats the point in modularisation? > > I'm arguing that it should be the module authors choice to apply the > tougher rules, but I'm very happy to hear other alternatives in the > problem-space. > > > There's also at least one improvement in the JDK worth > > considering, which a few of us have already discussed, namely a small > > enhancement to javac that would allow a single invocation to compile a > > module, in module mode [2], but target class files other than > > `module-info.class` to an earlier release [3]. > > +1 > > Stephen > From Alan.Bateman at oracle.com Tue Mar 27 08:29:36 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Mar 2018 09:29:36 +0100 Subject: The baby and the bathwater In-Reply-To: References: <20180326180838.7135C196C48@eggemoggin.niobe.net> Message-ID: <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> On 27/03/2018 08:15, C?dric Champeau wrote: > Dual testing is a minimum. In practice, it depends on the kind of tests. > Typically, before JDK 9 for unit tests you never needed a jar to execute > unit tests. Maven happens to built it, but in practice a class directory + > resources is enough This hasn't changed. You can put directories containing the test classes + resources on the class path as before. When testing modules you can patch a module to add the test classes (and resources) that are compiled into a directory, no need for either the module or the tests to be packaged as JAR files. Maybe your comment is about testing libraries that are Multi-Release JARs? -Alan. From cedric.champeau at gmail.com Tue Mar 27 08:44:40 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Tue, 27 Mar 2018 10:44:40 +0200 Subject: The baby and the bathwater In-Reply-To: <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> Message-ID: Yes, precisely. It's not because your library works when you use class directory + resources, that once packaged, it still works. And since there's a recommendation to use mrjars to package module-info.class in the case of a library targetting both classpath (pre java 9 and java 9) and module path (java 9+), it's important to check both. Before, you could afford unit testing only with the class directory variant, but that's not so evident anymore. 2018-03-27 10:29 GMT+02:00 Alan Bateman : > On 27/03/2018 08:15, C?dric Champeau wrote: > >> Dual testing is a minimum. In practice, it depends on the kind of tests. >> Typically, before JDK 9 for unit tests you never needed a jar to execute >> unit tests. Maven happens to built it, but in practice a class directory + >> resources is enough >> > This hasn't changed. You can put directories containing the test classes + > resources on the class path as before. When testing modules you can patch a > module to add the test classes (and resources) that are compiled into a > directory, no need for either the module or the tests to be packaged as JAR > files. > > Maybe your comment is about testing libraries that are Multi-Release JARs? > > -Alan. > From forax at univ-mlv.fr Tue Mar 27 09:04:25 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 27 Mar 2018 11:04:25 +0200 (CEST) Subject: The baby and the bathwater In-Reply-To: <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> Message-ID: <1313007240.690740.1522141465142.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Alan Bateman" > ?: "C?dric Champeau" > Cc: "jigsaw-dev" > Envoy?: Mardi 27 Mars 2018 10:29:36 > Objet: Re: The baby and the bathwater > On 27/03/2018 08:15, C?dric Champeau wrote: >> Dual testing is a minimum. In practice, it depends on the kind of tests. >> Typically, before JDK 9 for unit tests you never needed a jar to execute >> unit tests. Maven happens to built it, but in practice a class directory + >> resources is enough > This hasn't changed. You can put directories containing the test classes > + resources on the class path as before. When testing modules you can > patch a module to add the test classes (and resources) that are compiled > into a directory, no need for either the module or the tests to be > packaged as JAR files. with the limitation that you can not patch a module-info so if you have testing-only dependencies like JUnit and you want to run them in module-mode, you have to generate a jar. > > Maybe your comment is about testing libraries that are Multi-Release JARs? > > -Alan. R?mi From Alan.Bateman at oracle.com Tue Mar 27 09:38:19 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 27 Mar 2018 10:38:19 +0100 Subject: The baby and the bathwater In-Reply-To: <1313007240.690740.1522141465142.JavaMail.zimbra@u-pem.fr> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <3a09969e-5e6a-cbaf-fcd9-b33aca995ca5@oracle.com> <1313007240.690740.1522141465142.JavaMail.zimbra@u-pem.fr> Message-ID: <74f77d47-fe63-f8f1-999d-c6b952344265@oracle.com> On 27/03/2018 10:04, Remi Forax wrote: > : > with the limitation that you can not patch a module-info so if you have testing-only dependencies like JUnit and you want to run them in module-mode, you have to generate a jar. > The --add-reads option is used to augment the module to read junit or other modules that are only required when testing. It works with exploded modules. I can't think of any limitations or differences between exploded and packaged modules to explain your comment. Also C?dric's comment about putting the module-info.class into a versioned section of a MR JAR is not something that has been recommended here. He might be running into issues with class path scanning tools that can't handle v53.0 class files or assume "module-info" is a valid class name. Those same tools may still have issues with .class files in META-INF/versions of course. -Alan From russell.gold at oracle.com Tue Mar 27 12:11:19 2018 From: russell.gold at oracle.com (Russell Gold) Date: Tue, 27 Mar 2018 08:11:19 -0400 Subject: Accessing internals in Java 11 In-Reply-To: References: <32ed1dce-7ef2-83a3-101f-5b1dba1b440f@codefx.org> Message-ID: That would be true if it was easy to find the Javadoc for the class. I think the problem may be that tooling for MR jars is still all-but-nonexistent, and that classes like Unsafe aren?t accessible when using the latest JDK with the ?release switch. That, plus the much faster release schedule, means that we need to support a wider range of Java versions than ever before, but don?t have the tools to facilitate doing so. - Russ > On Mar 25, 2018, at 9:57 AM, Alan Bateman wrote: > > On 25/03/2018 08:32, Nicolai Parlog wrote: >> Hi! >> >> On Java 9 and 10, the JPMS is forgiving when it comes to illegal >> access of JDK internals and jdk.unsupported offers classes like Unsafe >> or Signal. The same is true for 11-b5. >> >> Are there any plans to change this, i.e. will Java 11 become stricter >> before the release? Would --illegal-access get a different default >> value or might jdk.unsupported get smaller? >> > TBD on dialing up the encapsulation of JDK internals, JDK 11 might be too soon. I'm sure there will be lots of discussion on this once the time comes. > > The only changes to class in the jdk.unsupported module so far have been the removal of Reflection.getClassClass and Unsafe.defineClass. There were terminally deprecated (@Deprecated forRemoval=true) so shouldn't be a surprise to anyone. > > -Alan From greggwon at cox.net Tue Mar 27 13:56:23 2018 From: greggwon at cox.net (Gregg Wonderly) Date: Tue, 27 Mar 2018 08:56:23 -0500 Subject: The baby and the bathwater In-Reply-To: References: Message-ID: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> I think that Stephen is largely announcing that JigSaw presents more problems than it solves for the community. My list of issues, which I have shared before, goes basically like this. 1. Modules should have versions which new tooling on community toolsets could use for managing details like maven is doing now, but without explicit version details available at runtime. There should be runtime version details available. Java?s dynamic binding mechanisms would allow a lot more flexibility in how software was assembled at runtime, and that would be a good thing. 2. The complete set of changes around JigSaw are primarily about isolating the community from the innards of the JDK implementation, which they developed reliance on due to the JCM not functioning and not caring enough about actual software development, but money making instead. Now, they have to completely rearchitect large amounts of software that is mostly 100% reliable, to use these JDK ?improvements?. What?s the chance of that happening for anything that is not going to be actively developed because it completely works as needed? 3. It seems from the outside, that there was little attention payed to the 80% of Java users who don?t write code or use enterprise services, but instead have Java applets, WebStart clients and clickable jars, until late in the game. This caused a derail of the projects desire to lock-it-down in JDK-9. Now that the realities of all of this are becoming much more obvious, it?s actually much clearer, that a lot of java code runs without deployment plans or explicit lifecycle management of neither the Java software nor the JDK version used to run it. The result is that more and more people are going to give up on using Java because they will try to use an application which has always worked for them, and it will either silently fail (clickable jar with a security violation), or scream about some kind of security violation (applet or webstart app) which will alarm them into deleting it or moving on. 4. When I was a member of the Sun Developer Advisory Council in the early 2000?s, I learned a lot about how Sun was focused on just the enterprise opportunity to sell hardware and support contracts. How did that work out? Oracle who is also a ?server? company, demonstrated again, with JigSaw, that only the enterprise platform environment with planned software schedules and controlled releases was interesting. In SDAC parties with James Gosling, we heard stories about his frustration with the desktop environment being ignored by Sun Management. There were a lot of ?I?m sorry? and ?I?m trying to work on those things? comments. Where?s Gosling now in Java involvement? 5. The AWT and Swing parts of Java as well as DOM integration with Applets hasn?t been touched or worked on in decades. The demise of the Netbeans team and other changes around various platforms and people who were serving communities outside of the enterprise platform, demonstrates that the platform is no longer viable for anything except enterprise class projects, and I?d suggest that even there, it?s questionable because of how these JDK changes are changing things that Oracle needs, but not ?providing? things that the community benefits from, except for Oracle?s argument that now the JDK can actually be improved. I know this sound extremely whiny and fussy. I understand how it can easily be ignored. The detail for me, is simply that after 20 years of Java, with the first 5-10 years being very exciting because of the opportunity for unifying platforms, we are now not doing that at all. Apple has moved on with Swift, so that platform is not interested in Java by and large. Windows has .Net, like it or not, and people use that platform because of the rich end to end integration it has with all aspects of the platform, unlike Java which is not completely focused on ignoring anything about the UX for desktop environments. Thus, we are left with ?Linux/BSD/Solaris? as the only place that Java is being used, to some degree. Python has a lot of traction because it is lighter weight and has better platform integration. It?s sad to say these things, because Java could of been so much more. But, instead, it?s getting to be less and less to fewer and fewer people, largely because the only focus is on the 10,000 customers who do enterprise computing, instead of the 10,000,000 developers who could really benefit from using Java. That was on a slide at the first SDAC meeting. Java was going to be for everyone! Gregg > On Mar 26, 2018, at 1:08 PM, mark.reinhold at oracle.com wrote: > > Stephen Colebourne's recent blog entry [1] contains many true statements, > along with some reasonable advice for library maintainers. To summarize: > > - As of Java 9, with Jigsaw, there are two ways in which a library can > be used: Either on the traditional class path, or on the newfangled > module path. If you maintain a library but don't modularize it then > it can still -- unbeknownst to you -- be used on the module path as > an automatic module. > > - When code runs on the module path there are some differences in the > behavior of some pre-9 APIs, in particular those related to resource > lookup. > > - As a consequence, if you maintain a library that's intended to work > on Java 9 or later then you should test it on both the class path > and the module path, even if you do nothing to convert your library > to a module. If your library doesn't work on the module path then > you should either fix it or document that limitation. > > - If you don't modularize your library, or at least claim an automatic > module name for it via the `Automatic-Module-Name` manifest entry, > then you potentially block the maintainers of libraries that depend > upon yours from modularizing their own libraries. > > - The tools that we use, and in particular Maven, could be improved. > It's difficult to compile the classes for a modular JAR file that's > intended to work on the class path of pre-9 releases, it's difficult > to test a library on both the class path and the module path, and > various Maven plugins still need to be upgraded to handle (or else > ignore) `module-info.java` files. (Stephen has helpfully filed > issues in the appropriate bug trackers for some of these problems.) > > - Some old tools, bytecode libraries, and other systems fail when they > encounter `module-info.class` files or multi-release JAR files. > > From these points Stephen concludes that the module system, "as currently > designed, has 'negative benefits' for open source libraries," saying that > this is primarily because "the split (bifurcation) of the module-path > from the class-path is an absolute nightmare." > > Hyperbole aside, Stephen's main complaint here is only about the need > to test a library on both the class path and the module path if it's > intended to work on Java 9 or later. With automated testing this > shouldn't, in principle, be a huge burden, but still it's worth asking > the question: Could we have avoided the need for such dual testing if > we hadn't introduced the module path as separate from the class path? > > Consider, as a thought experiment, an alternative Jigsaw design that > didn't have a separate module path, and instead treated modular JARs > on the class path as modules rather than traditional JAR files. You > wouldn't have to dual-test if your baseline is Java 9 or later, but > if you want to support earlier releases with the same artifact then > you'd still have to test on the class path. > > With the actual Jigsaw design you do need to dual-test your library > when your baseline is Java 9 or later. There is, however, a benefit > to this: If someone uses your library in an application that works on > the Java 8 class path today then they can migrate it to the Java 9 (or > later) class path and then, when they're ready, move your library (and > perhaps some others) over to the module path. (There were many other > reasons to define the module path as separate from the class path, but > those aren't directly relevant here.) > > The tradeoff, then, is a little bit more dual testing on the part of > library maintainers in exchange for greater flexibility for those who > will migrate existing applications to Java 9 or later releases. Many > library maintainers will be reluctant to baseline to Java 9 (or later) > for a while yet, so they'll be dual-testing anyway, so I think this > was the right tradeoff. > > Stephen closes with a specific suggestion: > > "There needs to be a way for a library author to insist that the > modular jar file they are producing can only be run on the module-path > (with any attempt to use it on the class-path preventing application > startup). This would eliminate the need for testing both class-path > and module-path." > > Yes, this would eliminate the need for dual testing, but only if you're > willing to baseline to Java 11. As with a unified class/module path, > however, if you want your library to work on earlier releases then you'd > also, still, have to test on the class path, and you'd make it harder for > application maintainers to migrate old class-path applications. I don't > think this idea is worth pursuing. > > What ideas are worth pursuing? We should, by all means, continue to > improve our tools, Jigsaw itself, and the rest of the JDK. Several of > us here collaborated on the initial support for modular development in > Maven, but clearly there's more to do. If nothing else, the Surefire > plugin should be able to test a library on both the class path and the > module path. There's also at least one improvement in the JDK worth > considering, which a few of us have already discussed, namely a small > enhancement to javac that would allow a single invocation to compile a > module, in module mode [2], but target class files other than > `module-info.class` to an earlier release [3]. > > To sum up: We have more work to do, based on practical experience. This > shouldn't be a surprise, with a change to the platform of this scope. > Let's keep doing that work, let's not unduly alarm ourselves or anyone > else, and please let's not throw out the baby with the bathwater. > > - Mark > > > [1] http://blog.joda.org/2018/03/jpms-negative-benefits.html > [2] http://openjdk.java.net/jeps/261#Compile-time > [3] https://bugs.openjdk.java.net/browse/JDK-8200254 From cedric.champeau at gmail.com Tue Mar 27 14:09:14 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Tue, 27 Mar 2018 16:09:14 +0200 Subject: The baby and the bathwater In-Reply-To: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: 2018-03-27 15:56 GMT+02:00 Gregg Wonderly : > I think that Stephen is largely announcing that JigSaw presents more > problems than it solves for the community. My list of issues, which I have > shared before, goes basically like this. > > 1. Modules should have versions which new tooling on community toolsets > could use for managing details like maven is doing now, but without > explicit version details available at runtime. There should be runtime > version details available. Java?s dynamic binding mechanisms would allow a > lot more flexibility in how software was assembled at runtime, and that > would be a good thing. > I'm not sure what you mean by "modules should have versions". They do, today, it's just not used, and I think it's a good thing. So I assume you are talking about enforcing requirements on versions in the module info file, in which case I strongly disagree. Disclaimer: I'm in the Gradle team. We think it's the build tool system (or runtime system when runtime plugins) do determine the right versions, given a set of constraints provided by the producer (the library author) and the consumer. Any real world application has version conflicts, and solving conflicts is *not* a trivial business, so you clearly don't want the JVM to do it. What if a library requires module X:1.0.1, but you discover a critical vulnerability in X? Should you upgrade to 1.0.2? Where do you find this information? Metadata is live, the only thing that the module should say, IMO, is precisely what it does today: "I require this module", but then delegate to the build tool the responsibility to find a version that works given all requirements/constraints (environment, target JDK, vulnerabilities, ...). I think what the JDK does today is the right balance. In particular, module-info is mostly focused on the runtime aspect, but there are clear differences between what you need for compile, for runtime, or to compile against a component. The constraints are not necessarily the same for all those, so they shouldn't be mixed. From njbartlett at gmail.com Tue Mar 27 14:58:07 2018 From: njbartlett at gmail.com (Neil Bartlett) Date: Tue, 27 Mar 2018 15:58:07 +0100 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: On Tue, Mar 27, 2018 at 3:09 PM, C?dric Champeau wrote: > 2018-03-27 15:56 GMT+02:00 Gregg Wonderly : > > > I think that Stephen is largely announcing that JigSaw presents more > > problems than it solves for the community. My list of issues, which I > have > > shared before, goes basically like this. > > > > 1. Modules should have versions which new tooling on community toolsets > > could use for managing details like maven is doing now, but without > > explicit version details available at runtime. There should be runtime > > version details available. Java?s dynamic binding mechanisms would > allow a > > lot more flexibility in how software was assembled at runtime, and that > > would be a good thing. > > > > I'm not sure what you mean by "modules should have versions". They do, > today, it's just not used, and I think it's a good thing. So I assume you > are talking about enforcing requirements on versions in the module info > file, in which case I strongly disagree. Disclaimer: I'm in the Gradle > team. We think it's the build tool system (or runtime system when runtime > plugins) do determine the right versions, given a set of constraints > provided by the producer (the library author) and the consumer. Any real > world application has version conflicts, and solving conflicts is *not* a > trivial business, so you clearly don't want the JVM to do it. What if a > library requires module X:1.0.1, but you discover a critical vulnerability > in X? Should you upgrade to 1.0.2?Where do you find this information? > Version ranges solve this problem. > Metadata is live, the only thing that the module should say, IMO, is > precisely what it does today: "I require this module", but then delegate to > the build tool the responsibility to find a version that works given all > requirements/constraints (environment, target JDK, vulnerabilities, ...). Whether or not JPMS enforces version constraints, the inability to even state a dependency upon a version of a module has created duplication.You have to state "require module" in module-info, and you have to repeat it with additional version information in the build descriptor (pom.xml, build.gradle, etc). If we could put a version requirement in module-info then the build tool could use that; this could have been achieved simply by permitting annotations on module-info 'require' statements. > I > think what the JDK does today is the right balance. In particular, > module-info is mostly focused on the runtime aspect, but there are clear > differences between what you need for compile, for runtime, or to compile > against a component. The constraints are not necessarily the same for all > those, so they shouldn't be mixed. > In what sense is module-info focused on the runtime aspect? It is enforced at both compile time and runtime, and yet it does not provide sufficient information for either the build tooling OR the runtime to assemble a consistent set of modules that work together. From cedric.champeau at gmail.com Tue Mar 27 15:07:54 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Tue, 27 Mar 2018 17:07:54 +0200 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: > Version ranges solve this problem. > They don't. They introduce new categories of problems (reproducibility, boundaries) and don't account for the fact that liveliness of a version is post-publication (a vulnerability is rarely discovered before releasing, typically). > Whether or not JPMS enforces version constraints, the inability to even > state a dependency upon a version of a module has created duplication.You > have to state "require module" in module-info, and you have to repeat it > with additional version information in the build descriptor (pom.xml, > build.gradle, etc). > You don't have to repeat. There's nothing that says that _you_ should write the module file. Also, the build tool _may_ source the dependencies from the module info file (but it wouldn't be enough, because you want different dependencies for test, compile, API, ...). Also a version is often misleading. What does it mean when you write "I depend on 1.0.4". Does it mean that it doesn't work on 1.0.3, or does it mean that it was the latest version that was available when you built? Or does it mean that actually this version is provided by your runtime environment, so it's a strict dependency? We're currently tackling all these problems, which are real world problems on medium to large scale applications. A single version number is often not enough: you need constraints, and sometimes variants (think classifiers). > > > > In what sense is module-info focused on the runtime aspect? It is enforced > at both compile time and runtime, and yet it does not provide sufficient > information for either the build tooling OR the runtime to assemble a > consistent set of modules that work together. > > The module info file defines the module graph, and is enforced at compile and runtime. However, it doesn't account for what you need: - when you build your library: API and implementation dependencies - when someone builds against your library: only API dependencies - when you run the library (API, implementation and "runtime only" dependencies) Nor does it know which of does are provided by the runtime environment, or compile tools. It only knows they are required, but barely knows who provides them, and for what use. From blackdrag at gmx.org Tue Mar 27 17:37:21 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Tue, 27 Mar 2018 19:37:21 +0200 Subject: The baby and the bathwater In-Reply-To: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: <71d4a5b7-efee-7835-690d-44a273212674@gmx.org> On 27.03.2018 15:56, Gregg Wonderly wrote: > I think that Stephen is largely announcing that JigSaw presents more problems than it solves for the community. My list of issues, which I have shared before, goes basically like this. > > 1. Modules should have versions which new tooling on community toolsets could use for managing details like maven is doing now, but without explicit version details available at runtime. There should be runtime version details available. Java?s dynamic binding mechanisms would allow a lot more flexibility in how software was assembled at runtime, and that would be a good thing. the module system is all about not dynamically assembling applications at runtime. JLink is the high point of that so far > 2. The complete set of changes around JigSaw are primarily about isolating the community from the innards of the JDK implementation, which they developed reliance on due to the JCM not functioning and not caring enough about actual software development, but money making instead. Now, they have to completely rearchitect large amounts of software that is mostly 100% reliable, to use these JDK ?improvements?. What?s the chance of that happening for anything that is not going to be actively developed because it completely works as needed? I still find comments like "not caring enough about actual software development, but money making instead" a bit unfair. Sure, those cases exist. But in other cases there used to be no other way, or there have been other needs that required doing a solution in a very stupid way. For example, for me to be able to call constructors of a super class I have to have an invokespecial. Since I have to decide what constructor to call at runtime I have basically something like a switch with multiple invokespecial and the switch-case deciding which one to take. This is a major hack, no Java compiler would ever emit code like that. But the Java compiler decides this at compile time, not my requirement at all. And of course when in Java8 the verifier got rewritten, this was no longer working for a while. But do you think there will be ever a better solution to this with these requirements? I doubt it. invokeSpecial has very specific semantics in this case, that are not going to change in my favor, just because I need that feature. The alternative is to introduce strange unrelated constructors, no longer enabling extending classes, wrapping everything, making constraints about the constructors... Alternatives are there... all horrible. I can give quite a list of things that are why they are because the software is almost 15 years old now. And I am pretty sure it would have never existed in a world of Java 9+... And actually the last point is what saddens me the most here. Anyway. You did go a way did does not fit jigsaw anymore, now you have to change the architecture - big times. So big, that there is no chance of hiding the changes from my users. And suddenly I will have a MOP that will have to expose do-what-you-want-with-me Lookup objects, just to get things somehow still working. And for those not actively developed it is easy. They will die at one point. Not that this is good, just the effect. > 3. It seems from the outside, that there was little attention payed to the 80% of Java users who don?t write code or use enterprise services, but instead have Java applets, WebStart clients and clickable jars, until late in the game. This caused a derail of the projects desire to lock-it-down in JDK-9. Now that the realities of all of this are becoming much more obvious, it?s actually much clearer, that a lot of java code runs without deployment plans or explicit lifecycle management of neither the Java software nor the JDK version used to run it. The result is that more and more people are going to give up on using Java because they will try to use an application which has always worked for them, and it will either silently fail (clickable jar with a security violation), or scream about some kind of security violation (applet or webstart app) which will alarm them into deleting it or moving on. enterprise services... considering how long jigsaw chose to ignore things like dependency injection and continues comments about how jigsaw is not for the enterprise, there is also a large portion of the enterprise services that got no attention. > 4. When I was a member of the Sun Developer Advisory Council in the early 2000?s, I learned a lot about how Sun was focused on just the enterprise opportunity to sell hardware and support contracts. How did that work out? Oracle who is also a ?server? company, demonstrated again, with JigSaw, that only the enterprise platform environment with planned software schedules and controlled releases was interesting. In SDAC parties with James Gosling, we heard stories about his frustration with the desktop environment being ignored by Sun Management. There were a lot of ?I?m sorry? and ?I?m trying to work on those things? comments. Where?s Gosling now in Java involvement? > 5. The AWT and Swing parts of Java as well as DOM integration with Applets hasn?t been touched or worked on in decades. The demise of the Netbeans team and other changes around various platforms and people who were serving communities outside of the enterprise platform, demonstrates that the platform is no longer viable for anything except enterprise class projects, and I?d suggest that even there, it?s questionable because of how these JDK changes are changing things that Oracle needs, but not ?providing? things that the community benefits from, except for Oracle?s argument that now the JDK can actually be improved. You forgot JavaFX. Since it will be no longer part of the JDK I did hear very worried words from some people and others giving up JavaFX completely. And no, they do not go to Swing and AWT. They move to javascript UIs. That means Java on the server if at all. Nashorn isn't even considered as an alternative to the likes of node and electron. Desktop Java as application, Applet or Webstart will have a very very bad standing in the future. The only reason to keep Java on the server and not use for example Go instead is... the supporting libraries... many of them having to do a lot of changes because of jigsaw. I have not worked on any application in the last 4 years that will work in that version on Java9 as module, nor will it do now. > I know this sound extremely whiny and fussy. I understand how it can easily be ignored. The detail for me, is simply that after 20 years of Java, with the first 5-10 years being very exciting because of the opportunity for unifying platforms, we are now not doing that at all. Apple has moved on with Swift, so that platform is not interested in Java by and large. Windows has .Net, like it or not, and people use that platform because of the rich end to end integration it has with all aspects of the platform, unlike Java which is not completely focused on ignoring anything about the UX for desktop environments. Thus, we are left with ?Linux/BSD/Solaris? as the only place that Java is being used, to some degree. Python has a lot of traction because it is lighter weight and has better platform integration. > > It?s sad to say these things, because Java could of been so much more. But, instead, it?s getting to be less and less to fewer and fewer people, largely because the only focus is on the 10,000 customers who do enterprise computing, instead of the 10,000,000 developers who could really benefit from using Java. That was on a slide at the first SDAC meeting. > > Java was going to be for everyone! But somebody has to pay the development. It is not like the JVM is a cash cow. But let me as a different question: Why should I use Java in the cloud? Oracle fails to answer this in my opinion. And in my opinion Oracle is on a good way to loose Java on the desktop as well as the server. I may overestimate trends I imagine to see of course. But frankly, the superior JVM technology and not having to compile for each platform (I ignore jlink here) are the things keeping me around for now. Java as platform would have to set a new positive trend to get out of this... difficult to do. bye Jochen From peter.levart at gmail.com Wed Mar 28 07:28:23 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 28 Mar 2018 09:28:23 +0200 Subject: The baby and the bathwater In-Reply-To: <20180326180838.7135C196C48@eggemoggin.niobe.net> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> Message-ID: <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> On 03/26/18 20:08, mark.reinhold at oracle.com wrote: > Stephen closes with a specific suggestion: > > "There needs to be a way for a library author to insist that the > modular jar file they are producing can only be run on the module-path > (with any attempt to use it on the class-path preventing application > startup). This would eliminate the need for testing both class-path > and module-path." That's easy to enforce in runtime. Just take a "victim" class from your library that is most often needed when your library is being used (or take a couple of them) and add a class initialization block like the following to them: public class Whatever { ??? static { ??? ??? if (Whatever.class.getModule().getName() == null) { ??? ??? ??? throw new Error("Can only use this library as a module"); ??? ??? } ??? } Regards, Peter From forax at univ-mlv.fr Wed Mar 28 07:41:26 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 28 Mar 2018 09:41:26 +0200 (CEST) Subject: The baby and the bathwater In-Reply-To: <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> Message-ID: <502166811.1142584.1522222886193.JavaMail.zimbra@u-pem.fr> yes, !Whatever.class.getModule().isNamed() R?mi ----- Mail original ----- > De: "Peter Levart" > ?: "mark reinhold" , "jigsaw-dev" > Envoy?: Mercredi 28 Mars 2018 09:28:23 > Objet: Re: The baby and the bathwater > On 03/26/18 20:08, mark.reinhold at oracle.com wrote: >> Stephen closes with a specific suggestion: >> >> "There needs to be a way for a library author to insist that the >> modular jar file they are producing can only be run on the module-path >> (with any attempt to use it on the class-path preventing application >> startup). This would eliminate the need for testing both class-path >> and module-path." > > That's easy to enforce in runtime. Just take a "victim" class from your > library that is most often needed when your library is being used (or > take a couple of them) and add a class initialization block like the > following to them: > > public class Whatever { > > ??? static { > ??? ??? if (Whatever.class.getModule().getName() == null) { > ??? ??? ??? throw new Error("Can only use this library as a module"); > ??? ??? } > ??? } > > > Regards, Peter From cedric.champeau at gmail.com Wed Mar 28 07:55:57 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Wed, 28 Mar 2018 09:55:57 +0200 Subject: The baby and the bathwater In-Reply-To: <502166811.1142584.1522222886193.JavaMail.zimbra@u-pem.fr> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> <502166811.1142584.1522222886193.JavaMail.zimbra@u-pem.fr> Message-ID: Although I doubt that making sure a library _only_ works on the module path is a good idea, would it make sense to introduce a helper method for this check? Something like: Module.assertOnModulePath() ? 2018-03-28 9:41 GMT+02:00 Remi Forax : > yes, > !Whatever.class.getModule().isNamed() > > R?mi > > ----- Mail original ----- > > De: "Peter Levart" > > ?: "mark reinhold" , "jigsaw-dev" < > jigsaw-dev at openjdk.java.net> > > Envoy?: Mercredi 28 Mars 2018 09:28:23 > > Objet: Re: The baby and the bathwater > > > On 03/26/18 20:08, mark.reinhold at oracle.com wrote: > >> Stephen closes with a specific suggestion: > >> > >> "There needs to be a way for a library author to insist that the > >> modular jar file they are producing can only be run on the > module-path > >> (with any attempt to use it on the class-path preventing application > >> startup). This would eliminate the need for testing both class-path > >> and module-path." > > > > That's easy to enforce in runtime. Just take a "victim" class from your > > library that is most often needed when your library is being used (or > > take a couple of them) and add a class initialization block like the > > following to them: > > > > public class Whatever { > > > > static { > > if (Whatever.class.getModule().getName() == null) { > > throw new Error("Can only use this library as a module"); > > } > > } > > > > > > Regards, Peter > From njbartlett at gmail.com Wed Mar 28 08:47:35 2018 From: njbartlett at gmail.com (Neil Bartlett) Date: Wed, 28 Mar 2018 09:47:35 +0100 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: On Tue, Mar 27, 2018 at 4:07 PM, C?dric Champeau wrote: > > Version ranges solve this problem. >> > > They don't. They introduce new categories of problems (reproducibility, > boundaries) and don't account for the fact that liveliness of a version is > post-publication (a vulnerability is rarely discovered before releasing, > typically). > They do. The fact that they create further (solvable) challenges does not mean that they are not a solution to the initial problem. The second point is also incorrect - indeed, this is exactly the point of using a range rather than a point version. If I depend on a range "[1.4, 2.0)", this is because I need a *feature* that was released in version 1.4 of the dependency. My module is compatible with version 1.4.0 and if there is a bugfix called 1.4.1 then my module is also compatible with that. My version range does not imply anything about vulnerabilities that may exist in version 1.4.0, and nobody is suggesting that it should. > > > >> Whether or not JPMS enforces version constraints, the inability to even >> state a dependency upon a version of a module has created duplication.You >> have to state "require module" in module-info, and you have to repeat it >> with additional version information in the build descriptor (pom.xml, >> build.gradle, etc). >> > > You don't have to repeat. There's nothing that says that _you_ should > write the module file. Also, the build tool _may_ source the dependencies > from the module info file (but it wouldn't be enough, because you want > different dependencies for test, compile, API, ...). > You keep making my point for me! Dependencies are different at compile time from runtime... not just in terms of versions but also identities. For example it's common practice to compile against a pure API but deploy with an implementation of the API. The module-info in JPMS, which is enforced at both compile time and runtime, works against that practical insight. > Also a version is often misleading. What does it mean when you write "I > depend on 1.0.4". > Well what does it mean when you write "compile 'foo:bar:1.0.4'" in your build.gradle file? It means you have compiled against that version of the API, and your module will be compatible with that version up to the next breaking change (2.0 if the dependency is using semver). If you wanted your module to be compatible with 1.0.3 then you would have compiled against 1.0.3. Unless you do that, there is no way for any tooling to infer that you are indeed compatible with 1.0.3. > Does it mean that it doesn't work on 1.0.3, or does it mean that it was > the latest version that was available when you built? Or does it mean that > actually this version is provided by your runtime environment, so it's a > strict dependency? We're currently tackling all these problems, which are > real world problems on medium to large scale applications. A single version > number is often not enough: you need constraints, and sometimes variants > (think classifiers). > > >> >> >> >> In what sense is module-info focused on the runtime aspect? It is >> enforced at both compile time and runtime, and yet it does not provide >> sufficient information for either the build tooling OR the runtime to >> assemble a consistent set of modules that work together. >> >> The module info file defines the module graph, and is enforced at compile > and runtime. However, it doesn't account for what you need: > > - when you build your library: API and implementation dependencies > - when someone builds against your library: only API dependencies > - when you run the library (API, implementation and "runtime only" > dependencies) > > Nor does it know which of does are provided by the runtime environment, or > compile tools. It only knows they are required, but barely knows who > provides them, and for what use. > If a module knows what it needs, it is not necessary to know "who" provides it. From cedric.champeau at gmail.com Wed Mar 28 09:01:04 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Wed, 28 Mar 2018 11:01:04 +0200 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: > > They do. The fact that they create further (solvable) challenges does not > mean that they are not a solution to the initial problem. > > The second point is also incorrect - indeed, this is exactly the point of > using a range rather than a point version. If I depend on a range "[1.4, > 2.0)", this is because I need a *feature* that was released in version 1.4 > of the dependency. > This is an arbitrary interpretation of a range. In practice people use them for very different purposes. If, when you write "[1.4,2.0)", you assume that you need a feature of 1.4, this is already an assumption. Most people use 1.4 as the baseline because _this is the latest version available when I started_. They also _assume_ that anything from 1.4 would work. In practice this is rarely the case. There are bugs introduced, there are binary incompatibilities (despite semantic versioning). Ranges are a _convenience_, but certainly not an answer. Ranges + locking are better, because you _have to_ test, but they don't account for the environment either (say, my app depends on servlet-api, which version should you use? It should be _strictly_ what the runtime environment will give). > > Well what does it mean when you write "compile 'foo:bar:1.0.4'" in your > build.gradle file? It means you have compiled against that version of the > API, and your module will be compatible with that version up to the next > breaking change (2.0 if the dependency is using semver). > > In Gradle you'd not use `compile` anymore. You would use: api 'foo:bar:1.0.4' for an API dependency, one that is exposed by your very own library (mostly maps to "requires transitive") and you'd use: implementation 'foo:baz:2.1.4' for an implementation dependency, that is _not_ exposed by your API (mostly maps to "requires"). And if a transitive dependency needs a different version, we have strategies to select a best match, or fail. > If you wanted your module to be compatible with 1.0.3 then you would have > compiled against 1.0.3. Unless you do that, there is no way for any tooling > to infer that you are indeed compatible with 1.0.3. > That's again an over simplification. Real world apps have different problems. You may say "1.0.3", because 1.0.2 had a bug. Maybe your app didn't even depend on the faulty behavior, but because it had a bug, you upgraded. And, maybe one of your dependencies actually required 1.0.2 because 1.0.3 introduced a regression. So you want to be able to downgrade dependencies. Gradle makes it possible. There's a big difference between the "ideal world", and the world we live in. From scolebourne at joda.org Wed Mar 28 09:18:27 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 28 Mar 2018 10:18:27 +0100 Subject: The baby and the bathwater In-Reply-To: <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> Message-ID: On 28 March 2018 at 08:28, Peter Levart wrote: > That's easy to enforce in runtime. Just take a "victim" class from your > library that is most often needed when your library is being used (or take a > couple of them) and add a class initialization block like the following to > them: > > public class Whatever { > > static { > if (Whatever.class.getModule().getName() == null) { > throw new Error("Can only use this library as a module"); > } > } Agreed that this has always been possible, but it is code not metadata. Really, it should be a startup JPMS error if the module isn't running in the expected mode. That way tools like Maven and Gradle can also take decisions based on the metadata. Stephen From njbartlett at gmail.com Wed Mar 28 09:22:40 2018 From: njbartlett at gmail.com (Neil Bartlett) Date: Wed, 28 Mar 2018 10:22:40 +0100 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: On Wed, Mar 28, 2018 at 10:01 AM, C?dric Champeau wrote: > They do. The fact that they create further (solvable) challenges does not >> mean that they are not a solution to the initial problem. >> >> The second point is also incorrect - indeed, this is exactly the point of >> using a range rather than a point version. If I depend on a range "[1.4, >> 2.0)", this is because I need a *feature* that was released in version 1.4 >> of the dependency. >> > > This is an arbitrary interpretation of a range. In practice people use > them for very different purposes. If, when you write "[1.4,2.0)", you > assume that you need a feature of 1.4, this is already an assumption. Most > people use 1.4 as the baseline because _this is the latest version > available when I started_. They also _assume_ that anything from 1.4 would > work. In practice this is rarely the case. There are bugs introduced, there > are binary incompatibilities (despite semantic versioning). Ranges are a > _convenience_, but certainly not an answer. Ranges + locking are better, > because you _have to_ test, but they don't account for the environment > either (say, my app depends on servlet-api, which version should you use? > It should be _strictly_ what the runtime environment will give). > You keep mixing up the perspective of an application (and application assembler) with the perspective of a library/module. As a library developer I should pick the lowest version of my dependencies that my library can build against. When assembling an application you pick the highest version of each module such that you have a graph that will resolve. Version ranges in a library indicate compatibility, they say nothing about buggy point versions of the dependency. Yes you still need a mechanism for locking buggy versions but you store that information outside the module descriptor (because we cannot know about buggy versions that may be released in the future). That locking mechanism is in the domain of application assembly. > > >> >> Well what does it mean when you write "compile 'foo:bar:1.0.4'" in your >> build.gradle file? It means you have compiled against that version of the >> API, and your module will be compatible with that version up to the next >> breaking change (2.0 if the dependency is using semver). >> >> > In Gradle you'd not use `compile` anymore. You would use: > > api 'foo:bar:1.0.4' > > for an API dependency, one that is exposed by your very own library > (mostly maps to "requires transitive") > > and you'd use: > > implementation 'foo:baz:2.1.4' > > for an implementation dependency, that is _not_ exposed by your API > (mostly maps to "requires"). > > And if a transitive dependency needs a different version, we have > strategies to select a best match, or fail. > Good to know. And how do you transform that information into module-info? You talked about generating module-info but it sounds like you would need two of them... one with foo.bar for compile (otherwise javac will barf) and the other with foo.baz for runtime (otherwise the runtime resolver will barf). > > >> If you wanted your module to be compatible with 1.0.3 then you would have >> compiled against 1.0.3. Unless you do that, there is no way for any tooling >> to infer that you are indeed compatible with 1.0.3. >> > > That's again an over simplification. Real world apps have different > problems. You may say "1.0.3", because 1.0.2 had a bug. Maybe your app > didn't even depend on the faulty behavior, but because it had a bug, you > upgraded. And, maybe one of your dependencies actually required 1.0.2 > because 1.0.3 introduced a regression. So you want to be able to downgrade > dependencies. Gradle makes it possible. There's a big difference between > the "ideal world", and the world we live in. > > > I love being told that I don't live in the real world, and that the problems I (and many others) have been solving for over a decade are insoluble :-) Downgrading with version ranges is of course possible so long as they are used properly, but they also protect you from downgrading SO far that you get problems like NoSuchMethodError, NCDFE, etc. From cedric.champeau at gmail.com Wed Mar 28 09:39:06 2018 From: cedric.champeau at gmail.com (=?UTF-8?Q?C=C3=A9dric_Champeau?=) Date: Wed, 28 Mar 2018 11:39:06 +0200 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: > > You keep mixing up the perspective of an application (and application > assembler) with the perspective of a library/module. > > It's interesting that you say this because we precisely value modeling applications and libraries differently, using different plugins. However not everybody does that and while we can think of _ideal_ ways to model things. the truth is that most people don't reason like that. They use Maven (or Ant), templates that generate projects for them, use `+` as their dependency versions, BOMs to "suggest versions". or think that there's no difference between a "compile" and "test" scope so we can put all dependencies in a single json file shared by the whole company. So we're not arguing about what the _ideal_ solution should be. We're arguing about the interpretation of version numbers, and what people expect. > > As a library developer I should pick the lowest version of my dependencies > that my library can build against. When assembling an application you pick > the highest version of each module such that you have a graph that will > resolve. > > Version ranges in a library indicate compatibility, they say nothing about > buggy point versions of the dependency. Yes you still need a mechanism for > locking buggy versions but you store that information outside the module > descriptor (because we cannot know about buggy versions that may be > released in the future). That locking mechanism is in the domain of > application assembly. > That's precisely the point. When, as a library author, you write: [1.0, 2.0), did you mean: - I tested all versions from 1.0 to 2.0 (excluded), and they work (they are _compatible_) or - I tested all versions from 1.0 to 1.4, because 1.4 was the latest available, and they work, and I suppose it's going to be true for anything up to 2.0 or - I tested with 1.0, and hopefully any higher version should work (most likely what you intend to say) or - You can build me with any version between 1.0 and 2, and it should compile fine or - You can build me with 1.0, and run with any superior version, should run fine - ... The reality is that 99% of developers don't make any difference between all this, they just took the latest version available, and built against it. > >> >> > > Good to know. And how do you transform that information into module-info? > You talked about generating module-info but it sounds like you would need > two of them... one with foo.bar for compile (otherwise javac will barf) and > the other with foo.baz for runtime (otherwise the runtime resolver will > barf). > Currently we don't do any generation. It's an option to do it, and I wouldn't say that it's the best one, I think it's still too soon to make the decision. Especially, module info contains _additional_ information, like services, that the build tool probably doesn't care about (again arguable, we could potentially model services too). Another option is to source dependencies from module-info, but it's not that simple (parsing, mapping to the appropriate configurations, ...). So while it's a bit annoying to have redundancy between dependencies declared in the build file and those declared in the module-info file, there *is* interest for both. In particular, what you build might not just be a single library. You might want to share dependencies between modules, and make sure they use the same versions. You might want to produce a platform definition (BOM) too. The things we produce are different from the things we need. > > I love being told that I don't live in the real world, and that the > problems I (and many others) have been solving for over a decade are > insoluble :-) > > It's not about not living in the real world or not. It's about the horrible truth of the hundreds of modules published on Maven Central that use hundreds of different conventions, both in versioning or publishing. And recognizing things like "you shouldn't have 2 slf4 bindings on your classpath". There's no silver bullet, so I don't think putting versions in module info would solve this, on the contrary, it would probably make things much harder for lots of people. From peter.levart at gmail.com Wed Mar 28 10:01:09 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 28 Mar 2018 12:01:09 +0200 Subject: The baby and the bathwater In-Reply-To: References: <20180326180838.7135C196C48@eggemoggin.niobe.net> <9cab02c8-90f7-3f78-d081-dc2d3f44c2ca@gmail.com> Message-ID: <5416c523-719b-8f72-8835-4a3af4b0f153@gmail.com> On 03/28/2018 11:18 AM, Stephen Colebourne wrote: > On 28 March 2018 at 08:28, Peter Levart wrote: >> That's easy to enforce in runtime. Just take a "victim" class from your >> library that is most often needed when your library is being used (or take a >> couple of them) and add a class initialization block like the following to >> them: >> >> public class Whatever { >> >> static { >> if (Whatever.class.getModule().getName() == null) { >> throw new Error("Can only use this library as a module"); >> } >> } > Agreed that this has always been possible, but it is code not > metadata. Really, it should be a startup JPMS error if the module > isn't running in the expected mode. That way tools like Maven and > Gradle can also take decisions based on the metadata. > > Stephen Hm, that would be hard to enforce, since what you're asking is for JPMS to issue an error when the library in question is not part of its search scope. Class-Path class loader could do that though (by searching for pa particular "sign" in the jars on the class path even before the JVM transfers control to MainClass.main()). But modulepath and classpath are not the only ways a particular jar can be deployed. It can be part of a .war or .ear. It can be deployed into a system that manages its own class loader(s), etc. All those mechanisms would have to agree on a "standard" way of specifying such metadata. Regards, Peter From dalibor.topic at oracle.com Wed Mar 28 11:17:53 2018 From: dalibor.topic at oracle.com (dalibor topic) Date: Wed, 28 Mar 2018 13:17:53 +0200 Subject: The baby and the bathwater In-Reply-To: References: <6732670B-05E4-43BA-86B0-1028B2071274@cox.net> Message-ID: On 28.03.2018 11:39, C?dric Champeau wrote: > It's not about not living in the real world or not. It's about the horrible > truth of the hundreds of modules published on Maven Central that use > hundreds of different conventions, both in versioning or publishing. A tangentially relevant paper in this context is http://homepages.dcc.ufmg.br/~mtov/pub/2017-saner-breaking-apis.pdf . Quoting from the abstract: "Our large scale analysis on 317 real-world Java libraries, 9K releases, and 260K client applications shows that (i) 14.78% of the API changes break compatibility with previous versions, (ii) the frequency of breaking changes increases over time, (iii) 2.54% of their clients are impacted, and (iv) systems with higher frequency of breaking changes are larger, more popular, and more active" cheers, dalibor topic -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From stephan.herrmann at berlin.de Thu Mar 29 14:36:24 2018 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Thu, 29 Mar 2018 16:36:24 +0200 Subject: Deprecated attribute in module-info.class? Message-ID: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. I do find a RuntimeVisibleAnnotation attribute representing @java.lang.Deprecated(since="9", forRemoval=true) but I don't find a Deprecated attribute. Is this intended? JVMS 4.7.15 neither includes nor excludes modules, but not generating the Deprecated attribute looks like an omission to me. best, Stephan PS: Tested also with JDK 10+46 From Alan.Bateman at oracle.com Thu Mar 29 16:02:51 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 29 Mar 2018 17:02:51 +0100 Subject: Deprecated attribute in module-info.class? In-Reply-To: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> Message-ID: <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> On 29/03/2018 15:36, Stephan Herrmann wrote: > I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. > > I do find a RuntimeVisibleAnnotation attribute representing > ? @java.lang.Deprecated(since="9", forRemoval=true) > but I don't find a Deprecated attribute. > > Is this intended? JVMS 4.7.15 neither includes nor excludes > modules, but not generating the Deprecated attribute looks > like an omission to me. Are you using `javap` to look at the class or something else? I ask because `javap` tool was updated in JDK 10 to render annotations in a more friendly way. When I use the JDK 10 `javap` to look at the jdk.xml.bind's module-info.class from JDK 9 I see: RuntimeVisibleAnnotations: ? 0: #5(#6=s#7,#8=Z#9) ??? java.lang.Deprecated( ????? since="9" ????? forRemoval=true ??? ) whereas the JDK 9 `javap` prints less useful output. -Alan From stephan.herrmann at berlin.de Thu Mar 29 22:49:04 2018 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 30 Mar 2018 00:49:04 +0200 Subject: Deprecated attribute in module-info.class? In-Reply-To: <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> Message-ID: On 29.03.2018 18:02, Alan Bateman wrote: > On 29/03/2018 15:36, Stephan Herrmann wrote: >> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. >> >> I do find a RuntimeVisibleAnnotation attribute representing >> ? @java.lang.Deprecated(since="9", forRemoval=true) >> but I don't find a Deprecated attribute. >> >> Is this intended? JVMS 4.7.15 neither includes nor excludes >> modules, but not generating the Deprecated attribute looks >> like an omission to me. > Are you using `javap` to look at the class or something else? > > I ask because `javap` tool was updated in JDK 10 to render annotations in a more friendly way. When I use the JDK 10 `javap` to look > at the jdk.xml.bind's module-info.class from JDK 9 I see: > > RuntimeVisibleAnnotations: > ? 0: #5(#6=s#7,#8=Z#9) > ??? java.lang.Deprecated( > ????? since="9" > ????? forRemoval=true > ??? ) > > whereas the JDK 9 `javap` prints less useful output. I did use JDK 9 javap, but was able to decipher the RuntimeVisibleAnnotations attribute as you can see above :) (Still good to know about the improvement in JDK 10). My question, however, concerns the lack of a Deprecated *attribute* a la JVMS 4.7.15. Is that intended or not? IOW, are compilers expected to evaluate RuntimeVisibleAnnotations in order to issue deprecation warnings? In the past this was not necessary. best, Stephan From jonathan.gibbons at oracle.com Thu Mar 29 22:56:11 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 29 Mar 2018 15:56:11 -0700 Subject: Deprecated attribute in module-info.class? In-Reply-To: References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> Message-ID: <66132931-1295-cf70-75d0-5228a46ba6f1@oracle.com> On 3/29/18 3:49 PM, Stephan Herrmann wrote: > On 29.03.2018 18:02, Alan Bateman wrote: >> On 29/03/2018 15:36, Stephan Herrmann wrote: >>> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. >>> >>> I do find a RuntimeVisibleAnnotation attribute representing >>> ? @java.lang.Deprecated(since="9", forRemoval=true) >>> but I don't find a Deprecated attribute. >>> >>> Is this intended? JVMS 4.7.15 neither includes nor excludes >>> modules, but not generating the Deprecated attribute looks >>> like an omission to me. >> Are you using `javap` to look at the class or something else? >> >> I ask because `javap` tool was updated in JDK 10 to render >> annotations in a more friendly way. When I use the JDK 10 `javap` to >> look at the jdk.xml.bind's module-info.class from JDK 9 I see: >> >> RuntimeVisibleAnnotations: >> ?? 0: #5(#6=s#7,#8=Z#9) >> ???? java.lang.Deprecated( >> ?????? since="9" >> ?????? forRemoval=true >> ???? ) >> >> whereas the JDK 9 `javap` prints less useful output. > > I did use JDK 9 javap, but was able to decipher the > RuntimeVisibleAnnotations attribute as you can see above :) > (Still good to know about the improvement in JDK 10). > > My question, however, concerns the lack of a Deprecated > *attribute* a la JVMS 4.7.15. > > Is that intended or not? > > IOW, are compilers expected to evaluate RuntimeVisibleAnnotations > in order to issue deprecation warnings? > In the past this was not necessary. > > best, > Stephan Stephan, Given JEP 277 [1], it is now necessary to read the annotation in order to determine if the annotated item has been deprecated for removal, and to act accordingly. -- Jon [1] http://openjdk.java.net/jeps/277 From stephan.herrmann at berlin.de Thu Mar 29 23:02:11 2018 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 30 Mar 2018 01:02:11 +0200 Subject: Deprecated attribute in module-info.class? In-Reply-To: <66132931-1295-cf70-75d0-5228a46ba6f1@oracle.com> References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> <66132931-1295-cf70-75d0-5228a46ba6f1@oracle.com> Message-ID: <7b20547b-844a-f271-6614-0b77e6eecd7a@berlin.de> On 30.03.2018 00:56, Jonathan Gibbons wrote: > On 3/29/18 3:49 PM, Stephan Herrmann wrote: > >> On 29.03.2018 18:02, Alan Bateman wrote: >>> On 29/03/2018 15:36, Stephan Herrmann wrote: >>>> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. >>>> >>>> I do find a RuntimeVisibleAnnotation attribute representing >>>> ? @java.lang.Deprecated(since="9", forRemoval=true) >>>> but I don't find a Deprecated attribute. >>>> >>>> Is this intended? JVMS 4.7.15 neither includes nor excludes >>>> modules, but not generating the Deprecated attribute looks >>>> like an omission to me. >>> Are you using `javap` to look at the class or something else? >>> >>> I ask because `javap` tool was updated in JDK 10 to render annotations in a more friendly way. When I use the JDK 10 `javap` to >>> look at the jdk.xml.bind's module-info.class from JDK 9 I see: >>> >>> RuntimeVisibleAnnotations: >>> ?? 0: #5(#6=s#7,#8=Z#9) >>> ???? java.lang.Deprecated( >>> ?????? since="9" >>> ?????? forRemoval=true >>> ???? ) >>> >>> whereas the JDK 9 `javap` prints less useful output. >> >> I did use JDK 9 javap, but was able to decipher the >> RuntimeVisibleAnnotations attribute as you can see above :) >> (Still good to know about the improvement in JDK 10). >> >> My question, however, concerns the lack of a Deprecated >> *attribute* a la JVMS 4.7.15. >> >> Is that intended or not? >> >> IOW, are compilers expected to evaluate RuntimeVisibleAnnotations >> in order to issue deprecation warnings? >> In the past this was not necessary. >> >> best, >> Stephan > > Stephan, > > Given JEP 277 [1], it is now necessary to read the annotation in order > to determine if the annotated item has been deprecated for removal, > and to act accordingly. That's true, I was just hoping, we could skip evaluating RuntimeVisibleAnnotations if no Deprecated attribute is found. Is using Deprecated attributes now deprecated ? :p Stephan From jonathan.gibbons at oracle.com Thu Mar 29 23:23:19 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 29 Mar 2018 16:23:19 -0700 Subject: Deprecated attribute in module-info.class? In-Reply-To: <7b20547b-844a-f271-6614-0b77e6eecd7a@berlin.de> References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> <66132931-1295-cf70-75d0-5228a46ba6f1@oracle.com> <7b20547b-844a-f271-6614-0b77e6eecd7a@berlin.de> Message-ID: <7baa7830-c356-ec3c-b69c-530e9a301939@oracle.com> On 3/29/18 4:02 PM, Stephan Herrmann wrote: > On 30.03.2018 00:56, Jonathan Gibbons wrote: >> On 3/29/18 3:49 PM, Stephan Herrmann wrote: >> >>> On 29.03.2018 18:02, Alan Bateman wrote: >>>> On 29/03/2018 15:36, Stephan Herrmann wrote: >>>>> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. >>>>> >>>>> I do find a RuntimeVisibleAnnotation attribute representing >>>>> ? @java.lang.Deprecated(since="9", forRemoval=true) >>>>> but I don't find a Deprecated attribute. >>>>> >>>>> Is this intended? JVMS 4.7.15 neither includes nor excludes >>>>> modules, but not generating the Deprecated attribute looks >>>>> like an omission to me. >>>> Are you using `javap` to look at the class or something else? >>>> >>>> I ask because `javap` tool was updated in JDK 10 to render >>>> annotations in a more friendly way. When I use the JDK 10 `javap` >>>> to look at the jdk.xml.bind's module-info.class from JDK 9 I see: >>>> >>>> RuntimeVisibleAnnotations: >>>> ?? 0: #5(#6=s#7,#8=Z#9) >>>> ???? java.lang.Deprecated( >>>> ?????? since="9" >>>> ?????? forRemoval=true >>>> ???? ) >>>> >>>> whereas the JDK 9 `javap` prints less useful output. >>> >>> I did use JDK 9 javap, but was able to decipher the >>> RuntimeVisibleAnnotations attribute as you can see above :) >>> (Still good to know about the improvement in JDK 10). >>> >>> My question, however, concerns the lack of a Deprecated >>> *attribute* a la JVMS 4.7.15. >>> >>> Is that intended or not? >>> >>> IOW, are compilers expected to evaluate RuntimeVisibleAnnotations >>> in order to issue deprecation warnings? >>> In the past this was not necessary. >>> >>> best, >>> Stephan >> >> Stephan, >> >> Given JEP 277 [1], it is now necessary to read the annotation in order >> to determine if the annotated item has been deprecated for removal, >> and to act accordingly. > > That's true, I was just hoping, we could skip evaluating > RuntimeVisibleAnnotations if no Deprecated attribute is found. > > Is using Deprecated attributes now deprecated ? :p > > Stephan > Stephan, Check these rules [1] which apply for class files with ACC_MODULE set: * |attributes|: One|Module|attribute must be present. Except for|Module|,|ModulePackages|,|ModuleMainClass|,|InnerClasses|,|SourceFile|,|SourceDebugExtension|,|RuntimeVisibleAnnotations|, and|RuntimeInvisibleAnnotations|, none of the pre-defined attributes (?4.7 ) may appear. -- Jon [1] https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-4.html#jvms-4.1 (see the end of this section) From stephan.herrmann at berlin.de Thu Mar 29 23:53:22 2018 From: stephan.herrmann at berlin.de (Stephan Herrmann) Date: Fri, 30 Mar 2018 01:53:22 +0200 Subject: Deprecated attribute in module-info.class? In-Reply-To: <7baa7830-c356-ec3c-b69c-530e9a301939@oracle.com> References: <0bbfab8f-6062-69bc-4334-600107f86d8c@berlin.de> <7db002f1-5a6d-4555-5dcd-dff1bed872fa@oracle.com> <66132931-1295-cf70-75d0-5228a46ba6f1@oracle.com> <7b20547b-844a-f271-6614-0b77e6eecd7a@berlin.de> <7baa7830-c356-ec3c-b69c-530e9a301939@oracle.com> Message-ID: <359cc5ed-1963-8a89-ea33-40603694a0d1@berlin.de> Thanks, Jon, that's exactly what I was looking for. Stephan On 30.03.2018 01:23, Jonathan Gibbons wrote: > > > On 3/29/18 4:02 PM, Stephan Herrmann wrote: >> On 30.03.2018 00:56, Jonathan Gibbons wrote: >>> On 3/29/18 3:49 PM, Stephan Herrmann wrote: >>> >>>> On 29.03.2018 18:02, Alan Bateman wrote: >>>>> On 29/03/2018 15:36, Stephan Herrmann wrote: >>>>>> I'm looking at jdk.xml.bind/module-info.class from JDK 9.0.4. >>>>>> >>>>>> I do find a RuntimeVisibleAnnotation attribute representing >>>>>> ? @java.lang.Deprecated(since="9", forRemoval=true) >>>>>> but I don't find a Deprecated attribute. >>>>>> >>>>>> Is this intended? JVMS 4.7.15 neither includes nor excludes >>>>>> modules, but not generating the Deprecated attribute looks >>>>>> like an omission to me. >>>>> Are you using `javap` to look at the class or something else? >>>>> >>>>> I ask because `javap` tool was updated in JDK 10 to render annotations in a more friendly way. When I use the JDK 10 `javap` to >>>>> look at the jdk.xml.bind's module-info.class from JDK 9 I see: >>>>> >>>>> RuntimeVisibleAnnotations: >>>>> ?? 0: #5(#6=s#7,#8=Z#9) >>>>> ???? java.lang.Deprecated( >>>>> ?????? since="9" >>>>> ?????? forRemoval=true >>>>> ???? ) >>>>> >>>>> whereas the JDK 9 `javap` prints less useful output. >>>> >>>> I did use JDK 9 javap, but was able to decipher the >>>> RuntimeVisibleAnnotations attribute as you can see above :) >>>> (Still good to know about the improvement in JDK 10). >>>> >>>> My question, however, concerns the lack of a Deprecated >>>> *attribute* a la JVMS 4.7.15. >>>> >>>> Is that intended or not? >>>> >>>> IOW, are compilers expected to evaluate RuntimeVisibleAnnotations >>>> in order to issue deprecation warnings? >>>> In the past this was not necessary. >>>> >>>> best, >>>> Stephan >>> >>> Stephan, >>> >>> Given JEP 277 [1], it is now necessary to read the annotation in order >>> to determine if the annotated item has been deprecated for removal, >>> and to act accordingly. >> >> That's true, I was just hoping, we could skip evaluating >> RuntimeVisibleAnnotations if no Deprecated attribute is found. >> >> Is using Deprecated attributes now deprecated ? :p >> >> Stephan >> > > Stephan, > > Check these rules [1] which apply for class files with ACC_MODULE set: > > ?* > > ?? |attributes|: One|Module|attribute must be present. Except > ?? for|Module|,|ModulePackages|,|ModuleMainClass|,|InnerClasses|,|SourceFile|,|SourceDebugExtension|,|RuntimeVisibleAnnotations|, > ?? and|RuntimeInvisibleAnnotations|, none of the pre-defined attributes > ?? (?4.7 > ?? ) > ?? may appear. > > -- Jon > > [1] https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-4.html#jvms-4.1 (see the end of this section) >