From tom.dewolf at aca-it.be Mon Jan 1 18:14:50 2018 From: tom.dewolf at aca-it.be (Tom De Wolf) Date: Mon, 01 Jan 2018 18:14:50 +0000 Subject: Versioning in the java platform module system - proof of concept Message-ID: I am working out a proof of concept to use the module version property and the version of a ?requires? in a useful manner. Details and a link to github can be found on: https://devcreativity.wordpress.com/2017/12/29/semantic-versions-in-java-9-modules-as-a-safety-net/ However, I stumbled on behaviour I don?t completely understand: - printing out the module.getDescription().version() showed a non-empty value for the jdk modules, explicit modules which got the version through ?jar --module-version?, and automatic modules like spring.core - as I understand the Requires.compiledVersion() gets a value when the dependency module has a version in module.getDescription().version() The latter is seen when requiring explicit modules, however requiring an automatic module does not fill in the compiledVersion? Why is that, even when the module.getDescription().version() shows a value (e.g. spring.core)? What is used for an automatic module such as spring.core to show 5.0.0.RELEASE as version? Thanks in advance for any helpful and insightful reply! Any thoughts, suggestions, remarks on my proof of concept itself are also more than welcome. Tom De Wolf From Alan.Bateman at oracle.com Tue Jan 2 08:06:05 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 08:06:05 +0000 Subject: Versioning in the java platform module system - proof of concept In-Reply-To: References: Message-ID: On 01/01/2018 18:14, Tom De Wolf wrote: > I am working out a proof of concept to use the module version property and > the version of a ?requires? in a useful manner. Details and a link to > github can be found on: > > https://devcreativity.wordpress.com/2017/12/29/semantic-versions-in-java-9-modules-as-a-safety-net/ > > However, I stumbled on behaviour I don?t completely understand: > - printing out the module.getDescription().version() showed a non-empty > value for the jdk modules, explicit modules which got the version through > ?jar --module-version?, and automatic modules like spring.core > - as I understand the Requires.compiledVersion() gets a value when the > dependency module has a version in module.getDescription().version() > > The latter is seen when requiring explicit modules, however requiring an > automatic module does not fill in the compiledVersion? Why is that, even > when the module.getDescription().version() shows a value (e.g. spring.core)? > > What is used for an automatic module such as spring.core to show > 5.0.0.RELEASE as version? > Requires::compiledVersion returns the module version if it is recorded in the module-info.class. javac has chosen to record the version of dependences when they are explicit modules. It doesn't record (or know anything about) the version of automatic modules at this time. It potentially could. Alternatively a build tool could record it (it doesn't have to be javac). -Alan From Alan.Bateman at oracle.com Tue Jan 2 08:13:43 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 08:13:43 +0000 Subject: reasons to add readability edges when creating layer In-Reply-To: <69a7398d-1c07-88d5-ec8a-b36a51f09448@poczta.onet.pl> References: <387ba3b6-7e5f-67dc-db33-b81e3e94c144@poczta.onet.pl> <91145b81-f83d-7183-9e19-1ffa981f289f@oracle.com> <69a7398d-1c07-88d5-ec8a-b36a51f09448@poczta.onet.pl> Message-ID: <0b98cecb-394a-fc45-c7db-c58183770b32@oracle.com> On 31/12/2017 16:06, Micha? Zegan wrote: > : > Is that planned to be added? Or even needed? I'm not aware of any proposals to add this in the short term. > I wanted to compare jpms > api to jboss-modules and seems to me that jboss-modules does not have > the dependency modification capability at all at the first glance. > Also, would the default module class loader be able to catch new read > edges if the target module was in a parent layer? Not sure how it maps > packages to modules... Neither the 3 built-in class loaders supporting the boot layer, nor the class loaders created by the defineModulesWithXXX methods, do not dynamically augment or change how they delegate. -Alan From tom.dewolf at aca-it.be Tue Jan 2 17:07:25 2018 From: tom.dewolf at aca-it.be (Tom De Wolf) Date: Tue, 02 Jan 2018 17:07:25 +0000 Subject: Versioning in the java platform module system - proof of concept In-Reply-To: References: Message-ID: Alan, Thx for the reply. If javac does it then it would be solved transparently in 1 place. What I don't understand is the reasoning why the module.getDescription().version() then returns a version for an automatic module like sprint.core? it is not an explicit module on which javac can do its magic, so where is that version information stored when there is no module-info? Thx Tom Op di 2 jan. 2018 om 09:06 schreef Alan Bateman : > On 01/01/2018 18:14, Tom De Wolf wrote: > > I am working out a proof of concept to use the module version property > and > > the version of a ?requires? in a useful manner. Details and a link to > > github can be found on: > > > > > https://devcreativity.wordpress.com/2017/12/29/semantic-versions-in-java-9-modules-as-a-safety-net/ > > > > However, I stumbled on behaviour I don?t completely understand: > > - printing out the module.getDescription().version() showed a non-empty > > value for the jdk modules, explicit modules which got the version through > > ?jar --module-version?, and automatic modules like spring.core > > - as I understand the Requires.compiledVersion() gets a value when the > > dependency module has a version in module.getDescription().version() > > > > The latter is seen when requiring explicit modules, however requiring an > > automatic module does not fill in the compiledVersion? Why is that, even > > when the module.getDescription().version() shows a value (e.g. > spring.core)? > > > > What is used for an automatic module such as spring.core to show > > 5.0.0.RELEASE as version? > > > Requires::compiledVersion returns the module version if it is recorded > in the module-info.class. javac has chosen to record the version of > dependences when they are explicit modules. It doesn't record (or know > anything about) the version of automatic modules at this time. It > potentially could. Alternatively a build tool could record it (it > doesn't have to be javac). > > -Alan > > From Alan.Bateman at oracle.com Tue Jan 2 17:31:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 17:31:04 +0000 Subject: Versioning in the java platform module system - proof of concept In-Reply-To: References: Message-ID: <9cc1f51b-4d60-a967-359f-bcbaeb6bfe06@oracle.com> On 02/01/2018 17:07, Tom De Wolf wrote: > : > > What I don't understand is the reasoning why the > module.getDescription().version() then returns a version for an > automatic module like sprint.core? it is not an explicit module on > which javac can do its magic, so where is that version information > stored when there is no module-info? The module version is derived from the name of the JAR file. If the JAR file is named spring-core-5.0.0.M1.jar then the module version will be "5.0.0.M1". All the details are in the ModuleFinder spec. javac doesn't derive a module version so there isn't a version string to record at compile-time. -Alan From david.lloyd at redhat.com Wed Jan 3 14:04:51 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 3 Jan 2018 08:04:51 -0600 Subject: reasons to add readability edges when creating layer In-Reply-To: <69a7398d-1c07-88d5-ec8a-b36a51f09448@poczta.onet.pl> References: <387ba3b6-7e5f-67dc-db33-b81e3e94c144@poczta.onet.pl> <91145b81-f83d-7183-9e19-1ffa981f289f@oracle.com> <69a7398d-1c07-88d5-ec8a-b36a51f09448@poczta.onet.pl> Message-ID: On Sun, Dec 31, 2017 at 10:06 AM, Micha? Zegan wrote: > Is that planned to be added? Or even needed? I wanted to compare jpms > api to jboss-modules and seems to me that jboss-modules does not have > the dependency modification capability at all at the first glance. JBoss Modules does allow modification of dependencies and even unloading of modules, however it must be done carefully (of course) with an understanding of how class loading works. This is not something that you will commonly want or need to do, because Java class linkage is a one-time operation. This means that if your module X depends on module A and a class from X links to a class form A, and you change the dependency from module A to module B, X will still have a connection to module A via the already-linked class. This is possibly the simplest example of where problems may begin. That said, the JBoss OSGi project uses this feature to implement its relinking capabilities. It mitigates the possible problems by way of its service container which helps to minimize the risk of problematic activity. -- - DML From webczat_200 at poczta.onet.pl Wed Jan 3 18:28:20 2018 From: webczat_200 at poczta.onet.pl (=?UTF-8?Q?Micha=c5=82_Zegan?=) Date: Wed, 3 Jan 2018 19:28:20 +0100 Subject: reasons to add readability edges when creating layer In-Reply-To: References: <387ba3b6-7e5f-67dc-db33-b81e3e94c144@poczta.onet.pl> <91145b81-f83d-7183-9e19-1ffa981f289f@oracle.com> <69a7398d-1c07-88d5-ec8a-b36a51f09448@poczta.onet.pl> Message-ID: Seems to me like a good argument for layers as sets of interdependent modules, as a module using another module prevents unloading the other module for real, as references still exist to it from the first module. Although I am not a module expert and trying to learn a bit about those things. As for jboss allowing dynamic dependency modification, I just haven't found that ability, but I have looked mainly at Module, ModuleLoader and such classes. W dniu 03.01.2018 o?15:04, David Lloyd pisze: > On Sun, Dec 31, 2017 at 10:06 AM, Micha? Zegan > wrote: >> Is that planned to be added? Or even needed? I wanted to compare jpms >> api to jboss-modules and seems to me that jboss-modules does not have >> the dependency modification capability at all at the first glance. > > JBoss Modules does allow modification of dependencies and even > unloading of modules, however it must be done carefully (of course) > with an understanding of how class loading works. > > This is not something that you will commonly want or need to do, > because Java class linkage is a one-time operation. This means that > if your module X depends on module A and a class from X links to a > class form A, and you change the dependency from module A to module B, > X will still have a connection to module A via the already-linked > class. This is possibly the simplest example of where problems may > begin. > > That said, the JBoss OSGi project uses this feature to implement its > relinking capabilities. It mitigates the possible problems by way of > its service container which helps to minimize the risk of problematic > activity. > From jonathan.gibbons at oracle.com Wed Jan 3 21:44:38 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 03 Jan 2018 13:44:38 -0800 Subject: RFR: JDK-8194141: Remove JDK9Wrappers Message-ID: <5A4D4EC6.8020606@oracle.com> Please review this code to remove a "temporary" class from javac, to provide reflective access to JDK 9 API, so that javac itself could be built with JDK 8. This temporary class is no longer required, now that the boot JDK for JDK 10 onwards is JDK 9. This is primarily about replacing the use of a javac internal class with the underlying direct API calls. However, a minor change to the autoconf spec is also required, to grant compile-time access to some internal JDK API. Some comments from the jdk.compiler module-info are also updated. The fix is proposed for JDK 11. JBS: https://bugs.openjdk.java.net/browse/JDK-8194141 Webrev: http://cr.openjdk.java.net/~jjg/8194141/webrev.00 -- Jon From jonathan.gibbons at oracle.com Wed Jan 3 22:42:18 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 03 Jan 2018 14:42:18 -0800 Subject: RFR: JDK-8194141: Remove JDK9Wrappers In-Reply-To: <5A4D4EC6.8020606@oracle.com> References: <5A4D4EC6.8020606@oracle.com> Message-ID: <5A4D5C4A.2000509@oracle.com> Updated webrev, reverting unnecessary change to generated-configure.sh http://cr.openjdk.java.net/~jjg/8194141/webrev.01/ -- Jon On 01/03/2018 01:44 PM, Jonathan Gibbons wrote: > Please review this code to remove a "temporary" class from javac, to > provide reflective access to JDK 9 API, so that javac itself could be > built with JDK 8. This temporary class is no longer required, now > that the boot JDK for JDK 10 onwards is JDK 9. > > This is primarily about replacing the use of a javac internal class > with the underlying direct API calls. However, a minor change to the > autoconf spec is also required, to grant compile-time access to some > internal JDK API. Some comments from the jdk.compiler module-info are > also updated. > > The fix is proposed for JDK 11. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8194141 > Webrev: http://cr.openjdk.java.net/~jjg/8194141/webrev.00 > > -- Jon > > From Alan.Bateman at oracle.com Thu Jan 4 07:53:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jan 2018 07:53:06 +0000 Subject: RFR: JDK-8194141: Remove JDK9Wrappers In-Reply-To: <5A4D5C4A.2000509@oracle.com> References: <5A4D4EC6.8020606@oracle.com> <5A4D5C4A.2000509@oracle.com> Message-ID: On 03/01/2018 22:42, Jonathan Gibbons wrote: > Updated webrev, reverting unnecessary change to generated-configure.sh > > http://cr.openjdk.java.net/~jjg/8194141/webrev.01/ This cleanup looks good to me. -Alan From erik.joelsson at oracle.com Thu Jan 4 08:33:14 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 4 Jan 2018 09:33:14 +0100 Subject: RFR: JDK-8194141: Remove JDK9Wrappers In-Reply-To: <5A4D5C4A.2000509@oracle.com> References: <5A4D4EC6.8020606@oracle.com> <5A4D5C4A.2000509@oracle.com> Message-ID: Looks good to me. /Erik On 2018-01-03 23:42, Jonathan Gibbons wrote: > Updated webrev, reverting unnecessary change to generated-configure.sh > > http://cr.openjdk.java.net/~jjg/8194141/webrev.01/ > > -- Jon > > On 01/03/2018 01:44 PM, Jonathan Gibbons wrote: >> Please review this code to remove a "temporary" class from javac, to >> provide reflective access to JDK 9 API, so that javac itself could be >> built with JDK 8.? This temporary class is no longer required, now >> that the boot JDK for JDK 10 onwards is JDK 9. >> >> This is primarily about replacing the use of a javac internal class >> with the underlying direct API calls. However, a minor change to the >> autoconf spec is also required, to grant compile-time access to some >> internal JDK API. Some comments from the jdk.compiler module-info are >> also updated. >> >> The fix is proposed for JDK 11. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8194141 >> Webrev: http://cr.openjdk.java.net/~jjg/8194141/webrev.00 >> >> -- Jon >> >> > From orionllmain at gmail.com Thu Jan 4 12:08:00 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Thu, 4 Jan 2018 19:08:00 +0700 Subject: Not all modules are listed in Javadoc Requires section Message-ID: I noticed that for some modules some dependencies are missing in the Javadoc Requires section. For example, `java.desktop` depends on `java.prefs`, `java.datatransfer`, `java.xml` but https://docs.oracle.com/javase/9/docs/api/java. desktop-summary.html says that it only depends on `java.datatransfer` on `java.xml`. Or `java.xml.ws` has 9 dependencies but only 3 transitive dependencies are listed: https://docs.oracle.com/javase/9/docs/api/java.xml.ws-summary.html Is this a bug? From Alan.Bateman at oracle.com Thu Jan 4 12:34:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jan 2018 12:34:39 +0000 Subject: Not all modules are listed in Javadoc Requires section In-Reply-To: References: Message-ID: <1a1a5555-5c05-bb56-7cd7-88c262f1d220@oracle.com> On 04/01/2018 12:08, Zheka Kozlov wrote: > I noticed that for some modules some dependencies are missing in the > Javadoc Requires section. > > For example, `java.desktop` depends on `java.prefs`, `java.datatransfer`, > `java.xml` but https://docs.oracle.com/javase/9/docs/api/java. > desktop-summary.html says that it only depends on `java.datatransfer` on > `java.xml`. > > Or `java.xml.ws` has 9 dependencies but only 3 transitive dependencies are > listed: https://docs.oracle.com/javase/9/docs/api/java.xml.ws-summary.html > > Is this a bug? No. The Java SE API docs are generated with `javadoc --expand-requires transitive`, not `javadoc --expand-requires all`. This means that the published docs include the modules that are `requires transitive`, it doesn't include the modules that are simply implementation dependencies (and don't contribute to the API). -Alan From nipa at codefx.org Thu Jan 4 14:18:33 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Thu, 4 Jan 2018 15:18:33 +0100 Subject: Qualified exports/opens and services Message-ID: Hi! First of all, happy new year, everybody. I got a question regarding qualified exports/opens and services. I thought I once read a proposal that a service consumer that some packages are exported or opened to could "forward" those privileges to the service providers. As an example, think about JPA and Hibernate (and ignore the ludicrous module names): * app code might `opens com.example.dto to jpa` * when app calls `EntityManager`, it locates Hibernate in module `hibernate` as JPA implementation * `hibernate` can now reflect over `com.example.dto` That feature doesn't exist, right? so long ... Nicolai -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From david.lloyd at redhat.com Thu Jan 4 14:25:27 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Thu, 4 Jan 2018 08:25:27 -0600 Subject: Qualified exports/opens and services In-Reply-To: References: Message-ID: On Thu, Jan 4, 2018 at 8:18 AM, Nicolai Parlog wrote: > Hi! > > First of all, happy new year, everybody. > > I got a question regarding qualified exports/opens and services. I > thought I once read a proposal that a service consumer that some > packages are exported or opened to could "forward" those privileges to > the service providers. > > As an example, think about JPA and Hibernate (and ignore the ludicrous > module names): > > * app code might `opens com.example.dto to jpa` > * when app calls `EntityManager`, it locates Hibernate in module > `hibernate` as JPA implementation > * `hibernate` can now reflect over `com.example.dto` > > That feature doesn't exist, right? Hi Nicolai, You may want to refer to the thread here: http://mail.openjdk.java.net/pipermail/jigsaw-dev//2017-November/013315.html Essentially a module may delegate a private Lookup to itself, or a module may do so on behalf of another module. To use your example, the module called "jpa" would have to internally create a private Lookup on "com.example.dto" and hand it over to the "hibernate" module, which would then be able to use it to reflect on private members of "com.example.dto". That said, the java.lang.invoke.MethodHandles#privateLookupIn() method leaves a little to be desired as it requires a class from the target module; it would be nice if there was a variant which accepted a Module... -- - DML From Alan.Bateman at oracle.com Thu Jan 4 15:06:32 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jan 2018 15:06:32 +0000 Subject: Qualified exports/opens and services In-Reply-To: References: Message-ID: <6cfde23e-49d1-cf88-049f-ba044794ae20@oracle.com> On 04/01/2018 14:25, David Lloyd wrote: > : > Hi Nicolai, > > You may want to refer to the thread here: > http://mail.openjdk.java.net/pipermail/jigsaw-dev//2017-November/013315.html > > Essentially a module may delegate a private Lookup to itself, or a > module may do so on behalf of another module. To use your example, > the module called "jpa" would have to internally create a private > Lookup on "com.example.dto" and hand it over to the "hibernate" > module, which would then be able to use it to reflect on private > members of "com.example.dto". I suspect Nicolai is looking for Module addOpens: "API Note: This method can be used for cases where a consumer module uses a qualified opens to open a package to an API module but where the reflective access to the members of classes in the consumer module is delegated to code in another module. Code in the API module can use this method to open the package in the consumer module to the other module." but you are right that using Lookup objects would work well here. No need to open any package, instead have the consumer module pass a Lookup object as a capability to the framework, the framework passes to the trusted service provider. > > That said, the java.lang.invoke.MethodHandles#privateLookupIn() method > leaves a little to be desired as it requires a class from the target > module; it would be nice if there was a variant which accepted a > Module... There wouldn't be a lookup class in that case so it would be a different kinda of lookup object. -Alan From nipa at codefx.org Thu Jan 4 22:14:56 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Thu, 4 Jan 2018 23:14:56 +0100 Subject: Qualified exports/opens and services In-Reply-To: <6cfde23e-49d1-cf88-049f-ba044794ae20@oracle.com> References: <6cfde23e-49d1-cf88-049f-ba044794ae20@oracle.com> Message-ID: Hi Alan, David, thanks for your replies, both are very helpful! :) so long ... Nicolai On 04.01.2018 16:06, Alan Bateman wrote: > On 04/01/2018 14:25, David Lloyd wrote: >> : Hi Nicolai, >> >> You may want to refer to the thread here: >> http://mail.openjdk.java.net/pipermail/jigsaw-dev//2017-November/013315.html >> >> >> >> Essentially a module may delegate a private Lookup to itself, or a >> module may do so on behalf of another module. To use your >> example, the module called "jpa" would have to internally create >> a private Lookup on "com.example.dto" and hand it over to the >> "hibernate" module, which would then be able to use it to reflect >> on private members of "com.example.dto". > I suspect Nicolai is looking for Module addOpens: > > "API Note: This method can be used for cases where a consumer > module uses a qualified opens to open a package to an API module > but where the reflective access to the members of classes in the > consumer module is delegated to code in another module. Code in the > API module can use this method to open the package in the consumer > module to the other module." > > but you are right that using Lookup objects would work well here. > No need to open any package, instead have the consumer module pass > a Lookup object as a capability to the framework, the framework > passes to the trusted service provider. > >> >> That said, the java.lang.invoke.MethodHandles#privateLookupIn() >> method leaves a little to be desired as it requires a class from >> the target module; it would be nice if there was a variant which >> accepted a Module... > There wouldn't be a lookup class in that case so it would be a > different kinda of lookup object. > > -Alan > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From nipa at codefx.org Thu Jan 4 22:16:58 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Thu, 4 Jan 2018 23:16:58 +0100 Subject: jar --print-module-descriptor Message-ID: <85075d5f-543e-7cff-efcb-888875161f42@codefx.org> Hi! The Java 9 jar documentation says[1]: -d or --print-module-descriptor Prints the module descriptor. It's --describe-module, though, right? At least that's what my 9.0.1 install says. so long ... Nicolai [1] https://docs.oracle.com/javase/9/tools/jar.htm#JSWOR614 -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From mandy.chung at oracle.com Thu Jan 4 23:03:01 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 4 Jan 2018 15:03:01 -0800 Subject: jar --print-module-descriptor In-Reply-To: <85075d5f-543e-7cff-efcb-888875161f42@codefx.org> References: <85075d5f-543e-7cff-efcb-888875161f42@codefx.org> Message-ID: I created https://bugs.openjdk.java.net/browse/JDK-8194653 to fix the docs bug.?? The docs was missed to be updated when this jar option was renamed to --describe-module. Thanks for reporting this. Mandy On 1/4/18 2:16 PM, Nicolai Parlog wrote: > Hi! > > The Java 9 jar documentation says[1]: > > -d or --print-module-descriptor > Prints the module descriptor. > > It's --describe-module, though, right? At least that's what my 9.0.1 > install says. > > so long ... Nicolai > > > [1] https://docs.oracle.com/javase/9/tools/jar.htm#JSWOR614 > > > From Alan.Bateman at oracle.com Fri Jan 5 07:23:33 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Jan 2018 07:23:33 +0000 Subject: jar --print-module-descriptor In-Reply-To: References: <85075d5f-543e-7cff-efcb-888875161f42@codefx.org> Message-ID: <4ba1aa40-0cc4-6511-9a22-7e4f1d9dd5b2@oracle.com> On 04/01/2018 23:03, mandy chung wrote: > I created https://bugs.openjdk.java.net/browse/JDK-8194653 to fix the > docs bug.?? The docs was missed to be updated when this jar option was > renamed to --describe-module. Here's the correct link. As Mandy says, this is an issue with the Oracle docs, the `jar --help` output lists the correct option name. https://bugs.openjdk.java.net/browse/JDK-8194654 From nipa at codefx.org Sat Jan 6 15:40:26 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sat, 6 Jan 2018 16:40:26 +0100 Subject: JDK created with jlink? Message-ID: Hi, I'm pretty sure I've read that jlink is used internally to create the canonical JRE/JDK runtime images. Did I get that right? so long ... Nicolai -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From Alan.Bateman at oracle.com Sat Jan 6 21:53:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 6 Jan 2018 21:53:14 +0000 Subject: JDK created with jlink? In-Reply-To: References: Message-ID: On 06/01/2018 15:40, Nicolai Parlog wrote: > Hi, > > I'm pretty sure I've read that jlink is used internally to create the > canonical JRE/JDK runtime images. Did I get that right? > Yes it does. You can look through Images.gmk [1] to see the options that it uses (if you are interested). -Alan [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/make/Images.gmk From nipa at codefx.org Sun Jan 7 09:14:33 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sun, 7 Jan 2018 10:14:33 +0100 Subject: JDK created with jlink? In-Reply-To: References: Message-ID: <22827b35-ae32-405b-30e1-f88d76af6576@codefx.org> Hi Alan, thank you for your answer. Your (and the entire team's) quick and informative responses are highly appreciated! :) so long ... Nicolai On 06.01.2018 22:53, Alan Bateman wrote: > > > On 06/01/2018 15:40, Nicolai Parlog wrote: >> ? Hi, >> >> I'm pretty sure I've read that jlink is used internally to create the >> canonical JRE/JDK runtime images. Did I get that right? >> > Yes it does. You can look through Images.gmk [1] to see the options that > it uses (if you are interested). > > -Alan > > [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/make/Images.gmk > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From nipa at codefx.org Sun Jan 7 12:57:51 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sun, 7 Jan 2018 13:57:51 +0100 Subject: Where do modules go in jlink images? Message-ID: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> Hi! Another question about jlink (guess what chapter I'm working on ;) ). I've tried to find out where jlink puts the content of the jmods folder, but hit a wall. I created a full runtime image with jlink and diffed the file trees (including disk usage), but all that got me was that my image doesn't contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and lib/plugin-legacy.jar - all of that is expected, but where are the class files from the jmod archives? Now I wonder, whether these are just ornamental - does lib/modules contain all the bytecode? I'm confused. so long ... Nicolai -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From forax at univ-mlv.fr Sun Jan 7 13:15:09 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 7 Jan 2018 14:15:09 +0100 (CET) Subject: Where do modules go in jlink images? In-Reply-To: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> Message-ID: <483385681.986885.1515330909797.JavaMail.zimbra@u-pem.fr> All classes are merged in the file named 'modules' and the native libraries are extracted from the jmods and put in the lib folder, because jlink create an image for a specific architecture/OS. cheers, R?mi ----- Mail original ----- > De: "Nicolai Parlog" > ?: "jigsaw-dev" > Envoy?: Dimanche 7 Janvier 2018 13:57:51 > Objet: Where do modules go in jlink images? > Hi! > > Another question about jlink (guess what chapter I'm working on ;) ). > I've tried to find out where jlink puts the content of the jmods > folder, but hit a wall. > > I created a full runtime image with jlink and diffed the file trees > (including disk usage), but all that got me was that my image doesn't > contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and > lib/plugin-legacy.jar - all of that is expected, but where are the > class files from the jmod archives? > > Now I wonder, whether these are just ornamental - does lib/modules > contain all the bytecode? I'm confused. > > so long ... Nicolai > > > > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > https://www.sitepoint.com/java > high-quality Java/JVM content > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx From nipa at codefx.org Sun Jan 7 13:26:08 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sun, 7 Jan 2018 14:26:08 +0100 Subject: Where do modules go in jlink images? In-Reply-To: <483385681.986885.1515330909797.JavaMail.zimbra@u-pem.fr> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> <483385681.986885.1515330909797.JavaMail.zimbra@u-pem.fr> Message-ID: <5be2cf06-309e-5779-d692-1d4f437001dd@codefx.org> Hi Remi, but the exact same thing happens when the JDK is linked with jlink, right? I just now see that the native binaries from the jmod archives are also present in lib/. So does that really mean that the jmods folder plays no role for the running JVM? What is it there for? so long ... Nicolai On 07.01.2018 14:15, Remi Forax wrote: > All classes are merged in the file named 'modules' and the native > libraries are extracted from the jmods and put in the lib folder, > because jlink create an image for a specific architecture/OS. > > cheers, R?mi > > ----- Mail original ----- >> De: "Nicolai Parlog" ?: "jigsaw-dev" >> Envoy?: Dimanche 7 Janvier 2018 >> 13:57:51 Objet: Where do modules go in jlink images? > >> Hi! >> >> Another question about jlink (guess what chapter I'm working on >> ;) ). I've tried to find out where jlink puts the content of the >> jmods folder, but hit a wall. >> >> I created a full runtime image with jlink and diffed the file >> trees (including disk usage), but all that got me was that my >> image doesn't contain bin/jmc, jmods, lib/misssioncontrol, >> lib/src.zip, and lib/plugin-legacy.jar - all of that is expected, >> but where are the class files from the jmod archives? >> >> Now I wonder, whether these are just ornamental - does >> lib/modules contain all the bytecode? I'm confused. >> >> so long ... Nicolai >> >> >> >> -- >> >> PGP Key: >> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 >> >> >> Web: >> http://codefx.org a blog about software development >> https://www.sitepoint.com/java high-quality Java/JVM content >> http://do-foss.de Free and Open Source Software for the City of >> Dortmund >> >> Twitter: https://twitter.com/nipafx > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From forax at univ-mlv.fr Sun Jan 7 13:38:19 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Sun, 7 Jan 2018 14:38:19 +0100 (CET) Subject: Where do modules go in jlink images? In-Reply-To: <5be2cf06-309e-5779-d692-1d4f437001dd@codefx.org> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> <483385681.986885.1515330909797.JavaMail.zimbra@u-pem.fr> <5be2cf06-309e-5779-d692-1d4f437001dd@codefx.org> Message-ID: <65469412.987667.1515332299261.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Nicolai Parlog" > ?: forax at univ-mlv.fr > Cc: "jigsaw-dev" > Envoy?: Dimanche 7 Janvier 2018 14:26:08 > Objet: Re: Where do modules go in jlink images? > Hi Remi, > > but the exact same thing happens when the JDK is linked with jlink, > right? I just now see that the native binaries from the jmod archives > are also present in lib/. > > So does that really mean that the jmods folder plays no role for the > running JVM? yes > What is it there for? make jlink working :) the transformation done by jlink is not reversible. > > so long ... Nicolai R?mi > > > > On 07.01.2018 14:15, Remi Forax wrote: >> All classes are merged in the file named 'modules' and the native >> libraries are extracted from the jmods and put in the lib folder, >> because jlink create an image for a specific architecture/OS. >> >> cheers, R?mi >> >> ----- Mail original ----- >>> De: "Nicolai Parlog" ?: "jigsaw-dev" >>> Envoy?: Dimanche 7 Janvier 2018 >>> 13:57:51 Objet: Where do modules go in jlink images? >> >>> Hi! >>> >>> Another question about jlink (guess what chapter I'm working on >>> ;) ). I've tried to find out where jlink puts the content of the >>> jmods folder, but hit a wall. >>> >>> I created a full runtime image with jlink and diffed the file >>> trees (including disk usage), but all that got me was that my >>> image doesn't contain bin/jmc, jmods, lib/misssioncontrol, >>> lib/src.zip, and lib/plugin-legacy.jar - all of that is expected, >>> but where are the class files from the jmod archives? >>> >>> Now I wonder, whether these are just ornamental - does >>> lib/modules contain all the bytecode? I'm confused. >>> >>> so long ... Nicolai >>> >>> >>> >>> -- >>> >>> PGP Key: >>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 >>> >>> >>> > Web: >>> http://codefx.org a blog about software development >>> https://www.sitepoint.com/java high-quality Java/JVM content >>> http://do-foss.de Free and Open Source Software for the City of >>> Dortmund >>> >>> Twitter: https://twitter.com/nipafx >> > > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > https://www.sitepoint.com/java > high-quality Java/JVM content > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx From nipa at codefx.org Sun Jan 7 13:49:54 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Sun, 7 Jan 2018 14:49:54 +0100 Subject: Where do modules go in jlink images? In-Reply-To: <65469412.987667.1515332299261.JavaMail.zimbra@u-pem.fr> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> <483385681.986885.1515330909797.JavaMail.zimbra@u-pem.fr> <5be2cf06-309e-5779-d692-1d4f437001dd@codefx.org> <65469412.987667.1515332299261.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi. >> So does that really mean that the jmods folder plays no role for >> the running JVM? > > yes Wow. > >> What is it there for? > > make jlink working :) the transformation done by jlink is not > reversible. Mind blown. Will have to go back to a previous chapter and do some correcting... so long ... Nicolai On 07.01.2018 14:38, forax at univ-mlv.fr wrote: > ----- Mail original ----- >> De: "Nicolai Parlog" ?: forax at univ-mlv.fr Cc: >> "jigsaw-dev" Envoy?: Dimanche 7 >> Janvier 2018 14:26:08 Objet: Re: Where do modules go in jlink >> images? > >> Hi Remi, >> >> but the exact same thing happens when the JDK is linked with >> jlink, right? I just now see that the native binaries from the >> jmod archives are also present in lib/. >> >> So does that really mean that the jmods folder plays no role for >> the running JVM? > > yes > >> What is it there for? > > make jlink working :) the transformation done by jlink is not > reversible. > >> >> so long ... Nicolai > > R?mi > >> >> >> >> On 07.01.2018 14:15, Remi Forax wrote: >>> All classes are merged in the file named 'modules' and the >>> native libraries are extracted from the jmods and put in the >>> lib folder, because jlink create an image for a specific >>> architecture/OS. >>> >>> cheers, R?mi >>> >>> ----- Mail original ----- >>>> De: "Nicolai Parlog" ?: "jigsaw-dev" >>>> Envoy?: Dimanche 7 Janvier >>>> 2018 13:57:51 Objet: Where do modules go in jlink images? >>> >>>> Hi! >>>> >>>> Another question about jlink (guess what chapter I'm working >>>> on ;) ). I've tried to find out where jlink puts the content >>>> of the jmods folder, but hit a wall. >>>> >>>> I created a full runtime image with jlink and diffed the >>>> file trees (including disk usage), but all that got me was >>>> that my image doesn't contain bin/jmc, jmods, >>>> lib/misssioncontrol, lib/src.zip, and lib/plugin-legacy.jar - >>>> all of that is expected, but where are the class files from >>>> the jmod archives? >>>> >>>> Now I wonder, whether these are just ornamental - does >>>> lib/modules contain all the bytecode? I'm confused. >>>> >>>> so long ... Nicolai >>>> >>>> >>>> >>>> -- >>>> >>>> PGP Key: >>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 >>>> >>>> >>>> >> >>>> Web: >>>> http://codefx.org a blog about software development >>>> https://www.sitepoint.com/java high-quality Java/JVM content >>>> http://do-foss.de Free and Open Source Software for the City >>>> of Dortmund >>>> >>>> Twitter: https://twitter.com/nipafx >>> >> >> -- >> >> PGP Key: >> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 >> >> >> Web: >> http://codefx.org a blog about software development >> https://www.sitepoint.com/java high-quality Java/JVM content >> http://do-foss.de Free and Open Source Software for the City of >> Dortmund >> >> Twitter: https://twitter.com/nipafx > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From tom at trellis.ch Sun Jan 7 17:23:16 2018 From: tom at trellis.ch (Thomas Brand) Date: Sun, 7 Jan 2018 18:23:16 +0100 Subject: Update a runtime image In-Reply-To: <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> References: <7b37b2a0-c648-6951-b0c3-c2ca8e11ec9a@oracle.com> <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> Message-ID: <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> On Sun, December 31, 2017 17:13, Remi Forax wrote: > jlink is not a distribution tool, you need to use a tool on top of jlink, > in the past, i've used the rpm format for that. > > We have stopped to distribute deltas when we have moved to docker that as > far as i know as no concept of incremental update. > > R?mi > > > ----- Mail original ----- > >> De: "Alan Bateman" >> ?: "Thomas Brand" , "jigsaw-dev" >> >> Envoy?: Dimanche 31 D?cembre 2017 16:55:37 >> Objet: Re: Update a runtime image >> > >> On 31/12/2017 15:42, Thomas Brand wrote: >> >>> Hello, I have a question related to jlink. >>> >>> >>> In a scenario where a runnable image created with jlink is installed, >>> and it should be updated, what are good strategies for doing this? >>> >>> I think if an update will introduce new dependencies this will be >>> different to updates that are known to be satisfied with what is >>> already in the image. >>> >>> The idea is to 'ship' just a small sort of diff / replacement part >>> that can be applied to the 'old' image to become 'new'. >>> >> The jlink tool always generates a new run-time image, there is no >> support at this time to install additional modules into a run-time >> image. >> >> -Alan. >> Thanks for the facts & suggestions! Greetings From Alan.Bateman at oracle.com Sun Jan 7 17:51:47 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 7 Jan 2018 17:51:47 +0000 Subject: Where do modules go in jlink images? In-Reply-To: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> Message-ID: <602fd444-39cd-1881-2a3a-9279ea18e8e1@oracle.com> On 07/01/2018 12:57, Nicolai Parlog wrote: > Hi! > > Another question about jlink (guess what chapter I'm working on ;) ). > I've tried to find out where jlink puts the content of the jmods > folder, but hit a wall. > > I created a full runtime image with jlink and diffed the file trees > (including disk usage), but all that got me was that my image doesn't > contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and > lib/plugin-legacy.jar - all of that is expected, but where are the > class files from the jmod archives? > > Now I wonder, whether these are just ornamental - does lib/modules > contain all the bytecode? I'm confused. > JEP 220 describes the layout of the run-time image and may help. Many modules will only contain classes and resources, they will go the jimage container (the internal "modules" file) as R?mi said. Many of the JDK modules have launchers (the jdk.compiler module has the javac launcher for example), native library, configuration files, legal notices, man pages, ...? The jlink tool puts these in the locations listed in the JEP. -Alan From nipa at codefx.org Mon Jan 8 20:06:09 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Mon, 8 Jan 2018 21:06:09 +0100 Subject: Where do modules go in jlink images? In-Reply-To: <602fd444-39cd-1881-2a3a-9279ea18e8e1@oracle.com> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> <602fd444-39cd-1881-2a3a-9279ea18e8e1@oracle.com> Message-ID: <51782631-8699-5edd-a1eb-a0554e2b0290@codefx.org> Hi Alan, my mistake was to assume that the JMOD files contain the bytecode that the JVM is running. A look into a JRE image would have saved me from that. so long ... Nicolai On 07.01.2018 18:51, Alan Bateman wrote: > On 07/01/2018 12:57, Nicolai Parlog wrote: >> ? Hi! >> >> Another question about jlink (guess what chapter I'm working on ;) ). >> I've tried to find out where jlink puts the content of the jmods >> folder, but hit a wall. >> >> I created a full runtime image with jlink and diffed the file trees >> (including disk usage), but all that got me was that my image doesn't >> contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and >> lib/plugin-legacy.jar - all of that is expected, but where are the >> class files from the jmod archives? >> >> Now I wonder, whether these are just ornamental - does lib/modules >> contain all the bytecode? I'm confused. >> > JEP 220 describes the layout of the run-time image and may help. Many > modules will only contain classes and resources, they will go the jimage > container (the internal "modules" file) as R?mi said. Many of the JDK > modules have launchers (the jdk.compiler module has the javac launcher > for example), native library, configuration files, legal notices, man > pages, ...? The jlink tool puts these in the locations listed in the JEP. > > -Alan > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From nipa at codefx.org Mon Jan 8 20:30:22 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Mon, 8 Jan 2018 21:30:22 +0100 Subject: Minimal and client VM for jlink? Message-ID: <67c4d3a8-e765-3e7d-186e-e26b7135db51@codefx.org> Hi, every source I found indicates that jlink makes it possible to create a runtime image with a client, server, or minimal VM. However I can't make it work for client and minimal: Error: Selected VM client doesn't exist. I tried with the Oracle JDK 9 for Linux (9 and 9.0.1) and the reference implementation[1]. It looks to me as if these JDKs only contain the server VM. Is that and the absence of client and minimal documented somewhere? Or am I just missing some flag? Is there a way to create an image with the minimal VM? so long ... Nicolai [1] http://jdk.java.net/java-se-ri/9 -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From david.holmes at oracle.com Mon Jan 8 21:29:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Jan 2018 07:29:39 +1000 Subject: Minimal and client VM for jlink? In-Reply-To: <67c4d3a8-e765-3e7d-186e-e26b7135db51@codefx.org> References: <67c4d3a8-e765-3e7d-186e-e26b7135db51@codefx.org> Message-ID: Hi Nicolai, On 9/01/2018 6:30 AM, Nicolai Parlog wrote: > Hi, > > every source I found indicates that jlink makes it possible to create a > runtime image with a client, server, or minimal VM. However I can't make > it work for client and minimal: > > Error: Selected VM client doesn't exist. > > I tried with the Oracle JDK 9 for Linux (9 and 9.0.1) and the reference > implementation[1]. It looks to me as if these JDKs only contain the > server VM. Is that and the absence of client and minimal documented > somewhere? Or am I just missing some flag? Is there a way to create an > image with the minimal VM? The Client and Minimal VMs are only available on 32-bit platforms. The 64-bit platforms do not have them. You may be able to change the build to force their creation with 64-bit but I'm not sure what the state of that is. For Oracle JDK the fact only the server VM exists is documented in the VM Guide: https://docs.oracle.com/javase/9/vm/java-virtual-machine-technology-overview.htm#JSJVM-GUID-982B244A-9B01-479A-8651-CB6475019281 "In Oracle Java Runtime Environment (JRE) 8 and earlier, different implementations of the JVM, (the client VM, server VM, and minimal VM), were supported for configurations commonly used as clients, servers, and for embedded systems. As most systems can now take advantage of the server VM, the Oracle Java Runtime Environment (JRE) 9 provides only that VM implementation." David > so long ... Nicolai > > > [1] http://jdk.java.net/java-se-ri/9 > > > From nipa at codefx.org Tue Jan 9 10:05:08 2018 From: nipa at codefx.org (Nicolai Parlog) Date: Tue, 9 Jan 2018 11:05:08 +0100 Subject: Minimal and client VM for jlink? In-Reply-To: References: <67c4d3a8-e765-3e7d-186e-e26b7135db51@codefx.org> Message-ID: Hi David, thank you very much. However much I search, I rarely end up finding this stuff myself (even "java se 9 vm client server minimal" doesn't do the trick). Maybe the jlink documentation and/or --list-plugins output could mention that, e.g. by adding that link? so long ... Nicolai On 08.01.2018 22:29, David Holmes wrote: > Hi Nicolai, > > On 9/01/2018 6:30 AM, Nicolai Parlog wrote: >> ? Hi, >> >> every source I found indicates that jlink makes it possible to create a >> runtime image with a client, server, or minimal VM. However I can't make >> it work for client and minimal: >> >> ????Error: Selected VM client doesn't exist. >> >> I tried with the Oracle JDK 9 for Linux (9 and 9.0.1) and the reference >> implementation[1]. It looks to me as if these JDKs only contain the >> server VM. Is that and the absence of client and minimal documented >> somewhere? Or am I just missing some flag? Is there a way to create an >> image with the minimal VM? > > The Client and Minimal VMs are only available on 32-bit platforms. The > 64-bit platforms do not have them. You may be able to change the build > to force their creation with 64-bit but I'm not sure what the state of > that is. > > For Oracle JDK the fact only the server VM exists is documented in the > VM Guide: > > https://docs.oracle.com/javase/9/vm/java-virtual-machine-technology-overview.htm#JSJVM-GUID-982B244A-9B01-479A-8651-CB6475019281 > > > "In Oracle Java Runtime Environment (JRE) 8 and earlier, different > implementations of the JVM, (the client VM, server VM, and minimal VM), > were supported for configurations commonly used as clients, servers, and > for embedded systems. As most systems can now take advantage of the > server VM, the Oracle Java Runtime Environment (JRE) 9 provides only > that VM implementation." > > David > >> ? so long ... Nicolai >> >> >> [1] http://jdk.java.net/java-se-ri/9 >> >> >> > -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development https://www.sitepoint.com/java high-quality Java/JVM content http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx From gunnar at hibernate.org Wed Jan 10 10:32:37 2018 From: gunnar at hibernate.org (Gunnar Morling) Date: Wed, 10 Jan 2018 11:32:37 +0100 Subject: Update a runtime image In-Reply-To: <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> References: <7b37b2a0-c648-6951-b0c3-c2ca8e11ec9a@oracle.com> <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> Message-ID: Hi Thomas, What I've found helpful is the ability to amend or override the set of modules contained in a modular runtime image via --module-path and --upgrade-module-path. If you use (Docker) containers to deploy your app, this can be employed nicely to separate images with your app's dependencies (as a modular runtime image) and a very thin image with your app itself. Unless your dependencies change, only the latter one needs to be re-built and distributed. The "exclude-resources" plug-in comes in handy for omitting your app from the modular runtime image. I've written about it here: https://github.com/moditect/moditect/wiki/Creating-dependency-images. --Gunnar 2018-01-07 18:23 GMT+01:00 Thomas Brand : > On Sun, December 31, 2017 17:13, Remi Forax wrote: > > jlink is not a distribution tool, you need to use a tool on top of jlink, > > in the past, i've used the rpm format for that. > > > > We have stopped to distribute deltas when we have moved to docker that as > > far as i know as no concept of incremental update. > > > > R?mi > > > > > > ----- Mail original ----- > > > >> De: "Alan Bateman" > >> ?: "Thomas Brand" , "jigsaw-dev" > >> > >> Envoy?: Dimanche 31 D?cembre 2017 16:55:37 > >> Objet: Re: Update a runtime image > >> > > > >> On 31/12/2017 15:42, Thomas Brand wrote: > >> > >>> Hello, I have a question related to jlink. > >>> > >>> > >>> In a scenario where a runnable image created with jlink is installed, > >>> and it should be updated, what are good strategies for doing this? > >>> > >>> I think if an update will introduce new dependencies this will be > >>> different to updates that are known to be satisfied with what is > >>> already in the image. > >>> > >>> The idea is to 'ship' just a small sort of diff / replacement part > >>> that can be applied to the 'old' image to become 'new'. > >>> > >> The jlink tool always generates a new run-time image, there is no > >> support at this time to install additional modules into a run-time > >> image. > >> > >> -Alan. > >> > > Thanks for the facts & suggestions! > Greetings > > > > > From gunnar at hibernate.org Wed Jan 10 10:37:35 2018 From: gunnar at hibernate.org (Gunnar Morling) Date: Wed, 10 Jan 2018 11:37:35 +0100 Subject: Where do modules go in jlink images? In-Reply-To: <51782631-8699-5edd-a1eb-a0554e2b0290@codefx.org> References: <03329c20-ad72-ced5-afc4-92a5f05a5d7d@codefx.org> <602fd444-39cd-1881-2a3a-9279ea18e8e1@oracle.com> <51782631-8699-5edd-a1eb-a0554e2b0290@codefx.org> Message-ID: Hi Nicolai, Funny, I was just exploring the related code the other day. If you are interested, look for usage of jdk.tools.jlink.internal.BasicImageWriter.MODULES_IMAGE_NAME, this will get you to the code for writing the "modules" file. --Gunnar 2018-01-08 21:06 GMT+01:00 Nicolai Parlog : > Hi Alan, > > my mistake was to assume that the JMOD files contain the bytecode that > the JVM is running. A look into a JRE image would have saved me from that. > > so long ... Nicolai > > > > On 07.01.2018 18:51, Alan Bateman wrote: > > On 07/01/2018 12:57, Nicolai Parlog wrote: > >> Hi! > >> > >> Another question about jlink (guess what chapter I'm working on ;) ). > >> I've tried to find out where jlink puts the content of the jmods > >> folder, but hit a wall. > >> > >> I created a full runtime image with jlink and diffed the file trees > >> (including disk usage), but all that got me was that my image doesn't > >> contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and > >> lib/plugin-legacy.jar - all of that is expected, but where are the > >> class files from the jmod archives? > >> > >> Now I wonder, whether these are just ornamental - does lib/modules > >> contain all the bytecode? I'm confused. > >> > > JEP 220 describes the layout of the run-time image and may help. Many > > modules will only contain classes and resources, they will go the jimage > > container (the internal "modules" file) as R?mi said. Many of the JDK > > modules have launchers (the jdk.compiler module has the javac launcher > > for example), native library, configuration files, legal notices, man > > pages, ... The jlink tool puts these in the locations listed in the JEP. > > > > -Alan > > > > -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > https://www.sitepoint.com/java > high-quality Java/JVM content > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx > From Alan.Bateman at oracle.com Wed Jan 10 15:28:46 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 Jan 2018 15:28:46 +0000 Subject: Update a runtime image In-Reply-To: References: <7b37b2a0-c648-6951-b0c3-c2ca8e11ec9a@oracle.com> <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> Message-ID: <0e03e7e0-86e2-ab0d-8c7b-b596f5aafddf@oracle.com> On 10/01/2018 10:32, Gunnar Morling wrote: > Hi Thomas, > > What I've found helpful is the ability to amend or override the set of > modules contained in a modular runtime image via --module-path and > --upgrade-module-path. > > If you use (Docker) containers to deploy your app, this can be employed > nicely to separate images with your app's dependencies (as a modular > runtime image) and a very thin image with your app itself. Unless your > dependencies change, only the latter one needs to be re-built and > distributed. The "exclude-resources" plug-in comes in handy for omitting > your app from the modular runtime image. I've written about it here: > https://github.com/moditect/moditect/wiki/Creating-dependency-images. > If I understand this correctly then the app module is linked into the run-time image to force its dependences to be linked in. It might be saner to just get its list of `requires` and specify that list to jlink. That way the app module can be specified on the module path (you are otherwise forced to use the upgrade module path to avoid the broken app module in the run-time image). -Alan From gunnar at hibernate.org Wed Jan 10 17:39:42 2018 From: gunnar at hibernate.org (Gunnar Morling) Date: Wed, 10 Jan 2018 18:39:42 +0100 Subject: Update a runtime image In-Reply-To: <0e03e7e0-86e2-ab0d-8c7b-b596f5aafddf@oracle.com> References: <7b37b2a0-c648-6951-b0c3-c2ca8e11ec9a@oracle.com> <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> <0e03e7e0-86e2-ab0d-8c7b-b596f5aafddf@oracle.com> Message-ID: 2018-01-10 16:28 GMT+01:00 Alan Bateman : > On 10/01/2018 10:32, Gunnar Morling wrote: > >> Hi Thomas, >> >> What I've found helpful is the ability to amend or override the set of >> modules contained in a modular runtime image via --module-path and >> --upgrade-module-path. >> >> If you use (Docker) containers to deploy your app, this can be employed >> nicely to separate images with your app's dependencies (as a modular >> runtime image) and a very thin image with your app itself. Unless your >> dependencies change, only the latter one needs to be re-built and >> distributed. The "exclude-resources" plug-in comes in handy for omitting >> your app from the modular runtime image. I've written about it here: >> https://github.com/moditect/moditect/wiki/Creating-dependency-images. >> >> If I understand this correctly then the app module is linked into the > run-time image to force its dependences to be linked in. It might be saner > to just get its list of `requires` and specify that list to jlink. That way > the app module can be specified on the module path (you are otherwise > forced to use the upgrade module path to avoid the broken app module in the > run-time image). What exactly do you mean by "broken app module"? It was my understanding that by applying that exclusion pattern, no trace whatsoever of the app module would end up in the resulting runtime image. If I run /bin/java --list-modules it's not shown in the image's module list, and things works for me when I add it via --module-path. But either what you suggest might be cleaner indeed. > > > -Alan > From Alan.Bateman at oracle.com Wed Jan 10 18:42:07 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 Jan 2018 18:42:07 +0000 Subject: Update a runtime image In-Reply-To: References: <7b37b2a0-c648-6951-b0c3-c2ca8e11ec9a@oracle.com> <1332365101.1176248.1514736805433.JavaMail.zimbra@u-pem.fr> <7c88d71b04a95bc21474be1839a86b4d.squirrel@ips73.ips.ch> <0e03e7e0-86e2-ab0d-8c7b-b596f5aafddf@oracle.com> Message-ID: <9006e77b-4585-a2d4-2fd4-18e6135efe19@oracle.com> On 10/01/2018 17:39, Gunnar Morling wrote: > : > > What exactly do you mean by "broken app module"? It was my > understanding that by applying that exclusion pattern, no trace > whatsoever of the app module would end up in the resulting runtime > image. If I run /bin/java --list-modules it's not shown in the > image's module list, and things works for me when I add it via > --module-path. I think there may be a bug here. The exclude-resources plugin was created for selectively removing resources, it was never mean to be able to remove a module completely. -Alan. From jeffrey_kutcher at yahoo.com Fri Jan 12 18:26:35 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Fri, 12 Jan 2018 18:26:35 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <974594798.1859150.1515765646023@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> Message-ID: <519854177.2043534.1515781595149@mail.yahoo.com> This is a good discussion but none of the suggestions satisfy the need I'm looking for.https://stackoverflow.com/questions/450807/how-do-i-make-the-method-return-type-generic The example code below shows button0 being added to a VBox. It also shows button1 being added to the same VBox only instead of being added directly to the list returned by vbox.getChildren(), that is vbox.getChildren().add(button1), buttton1 is added via reflection/introspection. The end goal is the same in both cases. Add a button. The methodology is different. Regardless, this should be trivial. It's not. Why? In this example, adding button0 produces no compile time warnings, Attempt-1. As for the code adding button1, Attempt-3 too produces no compile time warning, yet when run produces the WARNINGS shown at the bottom of the code. Commenting out Attempt-3 and uncommenting Attempt-2, produces compile time warnings. The bottom of the code example shows how to compile and run the example. As is, this code references no illegal reference code and yet still produces run time WARNINGS. Is there a way to access the list returned by getChildren() via reflection so a widget can be added without producing the run time WARNINGS? It's not the WARNINGS that I'm concerned about. I'm concerned that when future Java releases become available this code will fail to work. I would like to find a solution so that doesn't happen and this code continues to work. Maps have similar behavior. Try and use reflection to iterate over a Map, a trivial exercise, and it quickly becomes an impossible task. The internal class of the Map isn't public making it impossible to iterate through the list. I'm assuming that is what we are seeing here as well. Personally I believe internal classes should never be used. They seem to produce more problems than they solve. They are a quick, lazy fix that usually results in painful long term support. Besides, internal classes defeat write once, run anywhere. If it's internal, it's not being used anywhere else; which means the code has no useful value elsewhere; obviously false. That implies the code will be duplicated elsewhere to leverage it's functionality, meaning the code is now written twice, defeating the montra. Therefore, internal classes should never be used. In the same spirit, classes should never be private for the same reason. The nice thing is, convention easily satisfies this suggestion. Just because you can, doesn't mean you should. Java has been out for 23 years. Does a solution exist? I would like to see this example code work for future releases of Java. Is there a solution? Would you please point me at an example. Thanks ---------- BEGIN SOURCE ---------- public class ReflectAccessBug extends javafx.application.Application { public static void main(String[] args) { launch(args); } public void start(javafx.stage.Stage stage) { javafx.scene.control.Button button0 = new javafx.scene.control.Button(); javafx.scene.control.Button button1 = new javafx.scene.control.Button(); javafx.scene.layout.VBox vbox = new javafx.scene.layout.VBox(); javafx.event.EventHandler event = e -> { javafx.scene.control.Button b = (javafx.scene.control.Button)e.getSource(); System.err.println(b.getText() + " " + e); }; button0.setText("button0"); button0.setOnAction(event); button1.setText("button1"); button1.setOnAction(event); vbox.getChildren().add(button0); // Attempt-1 // This works as expected. //vbox.getChildren().add(button1); try { // Now let's try adding button1 using introspection ... java.lang.reflect.Method m = vbox.getClass().getMethod("getChildren", new Class[0]); Object o = m.invoke(vbox, new Object[0]); /* // Attempt-2 // This causes the same runtime warning as Attempt-3 // and I don't believe it should. // Attempt-2 produces a compile time warning. // Attempt-3 eliminates the compile time warning. javafx.collections.ObservableList l = (javafx.collections.ObservableList)o; m = l.getClass().getMethod("add", new Class[] { Object.class, }); o = m.invoke(l, new Object[] { button1, }); System.err.println("o="+o); */ // Attempt-3 // This results with the same runtime warning as Attempt-2. // I don't believe it should produce this warning and is the // point of this exercise. // Attempt-3 produces *NO* compile time warning as in Attempt-2. // It's not the compile time warning I'm concerned about. It's the // runtime warning that is the issue. // This code references no illegal reference code and yet still // produces a runtime warning. It references // "illegal reflective access operations". It's not this code // that is illegally accessing reflective code. Lower level code // accesses illegal reflective code on behalf of this code. m = o.getClass().getMethod("add", new Class[] { Object.class, }); o = m.invoke(o, new Object[] { button1, }); System.err.println("o="+o); } catch (Exception x) { x.printStackTrace(); } javafx.scene.Scene scene0 = new javafx.scene.Scene(vbox); stage.setScene(scene0); stage.show(); } } /* # compile: javac ReflectAccessBug.java # run: java ReflectAccessBug WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by ReflectAccessBug (...) to method com.sun.javafx.collections.VetoableListDecorator.add(java.lang.Object) WARNING: Please consider reporting this to the maintainers of ReflectAccessBug WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release */ ---------- END SOURCE ---------- From peter.levart at gmail.com Fri Jan 12 19:21:04 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 12 Jan 2018 20:21:04 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <519854177.2043534.1515781595149@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> Message-ID: <18b30ea8-07b0-9fd1-5d39-0164369bd890@gmail.com> Hi Jeffrey, See my answer inline... On 01/12/18 19:26, jeffrey kutcher wrote: > > This is a good discussion but none of the suggestions satisfy the need I'm looking for.https://stackoverflow.com/questions/450807/how-do-i-make-the-method-return-type-generic > > The example code below shows button0 being added to a VBox. It also shows button1 being added to the same VBox only instead of being added directly to the list returned by vbox.getChildren(), that is vbox.getChildren().add(button1), buttton1 is added via reflection/introspection. The end goal is the same in both cases. Add a button. The methodology is different. Regardless, this should be trivial. It's not. Why? > In this example, adding button0 produces no compile time warnings, Attempt-1. As for the code adding button1, Attempt-3 too produces no compile time warning, yet when run produces the WARNINGS shown at the bottom of the code. Commenting out Attempt-3 and uncommenting Attempt-2, produces compile time warnings. > The bottom of the code example shows how to compile and run the example. > As is, this code references no illegal reference code and yet still produces run time WARNINGS. > Is there a way to access the list returned by getChildren() via reflection so a widget can be added without producing the run time WARNINGS? It's not the WARNINGS that I'm concerned about. I'm concerned that when future Java releases become available this code will fail to work. I would like to find a solution so that doesn't happen and this code continues to work. > Maps have similar behavior. Try and use reflection to iterate over a Map, a trivial exercise, and it quickly becomes an impossible task. The internal class of the Map isn't public making it impossible to iterate through the list. I'm assuming that is what we are seeing here as well. > Personally I believe internal classes should never be used. They seem to produce more problems than they solve. They are a quick, lazy fix that usually results in painful long term support. Besides, internal classes defeat write once, run anywhere. If it's internal, it's not being used anywhere else; which means the code has no useful value elsewhere; obviously false. That implies the code will be duplicated elsewhere to leverage it's functionality, meaning the code is now written twice, defeating the montra. Therefore, internal classes should never be used. In the same spirit, classes should never be private for the same reason. The nice thing is, convention easily satisfies this suggestion. Just because you can, doesn't mean you should. > Java has been out for 23 years. Does a solution exist? I would like to see this example code work for future releases of Java. > Is there a solution? Would you please point me at an example. > Thanks > ---------- BEGIN SOURCE ---------- > > public class ReflectAccessBug extends javafx.application.Application { > > public static void main(String[] args) { > launch(args); > } > > public void start(javafx.stage.Stage stage) { > javafx.scene.control.Button button0 = new javafx.scene.control.Button(); > javafx.scene.control.Button button1 = new javafx.scene.control.Button(); > javafx.scene.layout.VBox vbox = new javafx.scene.layout.VBox(); > > javafx.event.EventHandler event = e -> { > javafx.scene.control.Button b = > (javafx.scene.control.Button)e.getSource(); > System.err.println(b.getText() + " " + e); > }; > > button0.setText("button0"); > button0.setOnAction(event); > > button1.setText("button1"); > button1.setOnAction(event); > > vbox.getChildren().add(button0); > > // Attempt-1 > // This works as expected. > //vbox.getChildren().add(button1); > > try { > // Now let's try adding button1 using introspection ... > java.lang.reflect.Method m = > vbox.getClass().getMethod("getChildren", new Class[0]); > Object o = m.invoke(vbox, new Object[0]); > /* > // Attempt-2 > // This causes the same runtime warning as Attempt-3 > // and I don't believe it should. > // Attempt-2 produces a compile time warning. > // Attempt-3 eliminates the compile time warning. > javafx.collections.ObservableList l = > (javafx.collections.ObservableList)o; > m = l.getClass().getMethod("add", new Class[] { Object.class, }); Try this instead of above line: m = javafx.collections.ObservableList.class.getMethod("add", new Class[] { Object.class, }); m.invoke(o, ....); The problem is that when you say: o.getClass().getMethod(....), you start searching for a method in the runtime implementation class of the object 'o' (it doesn't matter if you assign variable 'o' to 'l' with a cast - 'l' will point to the same runtime object as 'o'). The method might be found in some class/interface that is not public or not exported. Class#getMethod always returns the Method object describing the most specific method - the method declared in the most specific type (see the javadoc of getMethod() for more accurate description).? What you may get back from such call is a Method object describing the declaration of the method in the implementation class. Although such method is public, the declaring class (implementation class) might not be accessible (either not public or not exported). You better start searching for a method in some well known public type (ObservableList interface for example) that you know declares the method that is implemented or overridden in the runtime implementation class. You will get a different Method object, describing the abstract interface method, but when you invoke the method via this Method object, the invocation will be dispatched to the most specific implementation of that method because it obeys the virtual dispatch semantics. Reflection access checks are unfortunately performed with the declaring type/method described in the Method object and not with the method that the invocation actually dispatches to. Hope this helps. Regards, Peter > o = m.invoke(l, new Object[] { button1, }); > System.err.println("o="+o); > */ > // Attempt-3 > // This results with the same runtime warning as Attempt-2. > // I don't believe it should produce this warning and is the > // point of this exercise. > // Attempt-3 produces *NO* compile time warning as in Attempt-2. > // It's not the compile time warning I'm concerned about. It's the > // runtime warning that is the issue. > // This code references no illegal reference code and yet still > // produces a runtime warning. It references > // "illegal reflective access operations". It's not this code > // that is illegally accessing reflective code. Lower level code > // accesses illegal reflective code on behalf of this code. > m = o.getClass().getMethod("add", new Class[] { Object.class, }); > o = m.invoke(o, new Object[] { button1, }); > System.err.println("o="+o); > } catch (Exception x) { > x.printStackTrace(); > } > > javafx.scene.Scene scene0 = new javafx.scene.Scene(vbox); > stage.setScene(scene0); > stage.show(); > } > > } > > /* > # compile: > javac ReflectAccessBug.java > > # run: > java ReflectAccessBug > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by ReflectAccessBug (...) to method com.sun.javafx.collections.VetoableListDecorator.add(java.lang.Object) > WARNING: Please consider reporting this to the maintainers of ReflectAccessBug > WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations > WARNING: All illegal access operations will be denied in a future release > */ > > ---------- END SOURCE ---------- > From mandy.chung at oracle.com Fri Jan 12 19:25:20 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 12 Jan 2018 11:25:20 -0800 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <519854177.2043534.1515781595149@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> Message-ID: <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> On 1/12/18 10:26 AM, jeffrey kutcher wrote: > m = o.getClass().getMethod("add", new Class[] { Object.class, }); > o = m.invoke(o, new Object[] { button1, }); o.getClass().getMethod(...) is an anti-pattern for finding a public method.?? Object.getClass() returns the implementation class while you want to invoke a public method `javafx.collections.ObservableList::add` in this case.? In this case, the declaring class of the method is known and so one way to fix it is to use the specific Class: Class observableListClass = javafx.collections.ObservableList.class; m = observableListClass.getMethod("add",new Class[] { Object.class, }); o = m.invoke(o,new Object[] { button1, }); Mandy From webczat_200 at poczta.onet.pl Fri Jan 12 19:37:20 2018 From: webczat_200 at poczta.onet.pl (=?UTF-8?Q?Micha=c5=82_Zegan?=) Date: Fri, 12 Jan 2018 20:37:20 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> Message-ID: Actually, shouldn't reflection be fixed so that you can invoke any method of a non exported class, if the method overrides one that is accessible to the class calling invoke... Not sure it makes sense... W dniu 12.01.2018 o?20:25, mandy chung pisze: > > > On 1/12/18 10:26 AM, jeffrey kutcher wrote: >> ???????????? m = o.getClass().getMethod("add", new Class[] { >> Object.class, }); >> ???????????? o = m.invoke(o, new Object[] { button1, }); > > o.getClass().getMethod(...) is an anti-pattern for finding a public > method.?? Object.getClass() returns the implementation class while you > want to invoke a public method `javafx.collections.ObservableList::add` > in this case.? In this case, the declaring class of the method is known > and so one way to fix it is to use the specific Class: > > Class observableListClass = javafx.collections.ObservableList.class; > m = observableListClass.getMethod("add",new Class[] { Object.class, }); > o = m.invoke(o,new Object[] { button1, }); > > Mandy > > > > From jeffrey_kutcher at yahoo.com Fri Jan 12 21:17:20 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Fri, 12 Jan 2018 21:17:20 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> Message-ID: <412824504.2154912.1515791840103@mail.yahoo.com> The idea is not to be specific, but to be general. This is an example. In practice, I don't know what "o" is and so there has to be a discovery phase. I'm not sure how to make it more general than discovering the class via o.getClass(). As in this example, yes the class object returned is an?ObservableList but I'm unable to add an object to this list via reflection. Why? And what can be done to change that? Why should it be changed? Because an object can be added to the list via vbox.getChildren().add(). Therefore an object should be permitted to be added via introspection. The avenue for the addition is different, the end result should be the same. There are lots of other classes in the Java libraries that have this same issue.? There's also the issue of what gets returned can be many different types. How do you figure out which type should be returned? Not a trivial question; not so sure there's an answer. In this case, all I need is a List. Not an ArrayList, or an ObservableList, LinkedList, SortedList .... Please don't focus on this ... yet. Original issue first. On Friday, January 12, 2018, 1:25:13 PM CST, mandy chung wrote: On 1/12/18 10:26 AM, jeffrey kutcher wrote: m = o.getClass().getMethod("add", new Class[] { Object.class, }); o = m.invoke(o, new Object[] { button1, }); o.getClass().getMethod(...) is an anti-pattern for finding a public method.?? Object.getClass() returns the implementation class while you want to invoke a public method `javafx.collections.ObservableList::add` in this case.? In this case, the declaring class of the method is known and so one way to fix it is to use the specific Class: Class observableListClass = javafx.collections.ObservableList.class; m = observableListClass.getMethod("add", new Class[] { Object.class, }); o = m.invoke(o, new Object[] { button1, }); Mandy From webczat_200 at poczta.onet.pl Fri Jan 12 21:49:47 2018 From: webczat_200 at poczta.onet.pl (=?UTF-8?Q?Micha=c5=82_Zegan?=) Date: Fri, 12 Jan 2018 22:49:47 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <412824504.2154912.1515791840103@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> Message-ID: <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> well you rather know what getChildren returns, so you know what the returned object is, like generally. W dniu 12.01.2018 o?22:17, jeffrey kutcher pisze: > The idea is not to be specific, but to be general. This is an example. In practice, I don't know what "o" is and so there has to be a discovery phase. I'm not sure how to make it more general than discovering the class via o.getClass(). As in this example, yes the class object returned is an?ObservableList but I'm unable to add an object to this list via reflection. Why? And what can be done to change that? Why should it be changed? Because an object can be added to the list via vbox.getChildren().add(). Therefore an object should be permitted to be added via introspection. The avenue for the addition is different, the end result should be the same. > There are lots of other classes in the Java libraries that have this same issue.? > There's also the issue of what gets returned can be many different types. How do you figure out which type should be returned? Not a trivial question; not so sure there's an answer. In this case, all I need is a List. Not an ArrayList, or an ObservableList, LinkedList, SortedList .... Please don't focus on this ... yet. Original issue first. > On Friday, January 12, 2018, 1:25:13 PM CST, mandy chung wrote: > > > > On 1/12/18 10:26 AM, jeffrey kutcher wrote: > > m = o.getClass().getMethod("add", new Class[] { Object.class, }); > o = m.invoke(o, new Object[] { button1, }); > > o.getClass().getMethod(...) is an anti-pattern for finding a public method.?? Object.getClass() returns the implementation class while you want to invoke a public method `javafx.collections.ObservableList::add` in this case.? In this case, the declaring class of the method is known and so one way to fix it is to use the specific Class: > Class observableListClass = javafx.collections.ObservableList.class; > m = observableListClass.getMethod("add", new Class[] { Object.class, }); > o = m.invoke(o, new Object[] { button1, }); > > Mandy > > > > > From blackdrag at gmx.org Mon Jan 15 14:42:52 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Mon, 15 Jan 2018 15:42:52 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> Message-ID: Am 12.01.2018 um 22:49 schrieb Micha? Zegan: > well you rather know what getChildren returns, so you know what the > returned object is, like generally. but as soon as the method you want to call is not on the statically declared type, nor within a set of known statically types you are having a problem. There is no good way (good in terms of usability and performance) to get an arbitrary valid method object without static information. You are always forced to traverse the hierarchy to find the most accessible type. You will have to check access... and then of course there the two bad scenarios where invocation and search are not the same class or actual caller and caller are not equal. Which leads to a lengthy search using reflection to find a method, transfer it to a method handle and then let the invoker do the actual call. bye Jochen From jeffrey_kutcher at yahoo.com Mon Jan 15 18:27:19 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Mon, 15 Jan 2018 18:27:19 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> Message-ID: <1152555021.3613809.1516040839387@mail.yahoo.com> Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be. You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated. Introspection of?vbox.getChildren().add(x) should be just as simple as calling?vbox.getChildren().add(x) itself, otherwise why have introspection at all? Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system. I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances. Looking for an Oracle to glean insight (implications of a lock down) I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work. I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation. It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever. I'm sure there are others. On Monday, January 15, 2018, 8:44:46 AM CST, Jochen Theodorou wrote: Am 12.01.2018 um 22:49 schrieb Micha? Zegan: > well you rather know what getChildren returns, so you know what the > returned object is, like generally. but as soon as the method you want to call is not on the statically declared type, nor within a set of known statically types you are having a problem. There is no good way (good in terms of usability and performance) to get an arbitrary valid method object without static information. You are always forced to traverse the hierarchy to find the most accessible type. You will have to check access... and then of course there the two bad scenarios where invocation and search are not the same class or actual caller and caller are not equal. Which leads to a lengthy search using reflection to find a method, transfer it to a method handle and then let the invoker do the actual call. bye Jochen From Alan.Bateman at oracle.com Mon Jan 15 19:24:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 15 Jan 2018 19:24:24 +0000 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1152555021.3613809.1516040839387@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> Message-ID: <8813d25e-4bdb-8f90-0f8a-2982921c45cc@oracle.com> On 15/01/2018 18:27, jeffrey kutcher wrote: > I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. It's always been possible for getClass() to return a non-public class and for the Method invoke in the code example to fail with IllegalAccessException. You may have just got lucky in the past.? At some point then I would expect the JavaFX modules to not be open by default so maybe now is the time to fix the issues. You can run with `--illegal-access=warn` to help track down other code that may be accidentally trying to access members of JDK internal classes.? You can run with `--illegal-access=deny` to see how that code behaves when the classes in the JDK modules are not open for illegal access. -Alan From Rony.Flatscher at wu.ac.at Mon Jan 15 20:56:51 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Mon, 15 Jan 2018 21:56:51 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? Message-ID: Finally having gained enough time to start rewriting the reflection part for a bridge between a scripting language (ooRexx) and Java 9. >From past discussions on this list my view upon accessing members in superclasses that are protected in Java 9 is possible as such protected members are regarded to be public (they are accessible) from their subclasses. To test against the new module system I have created three simple modules (only what is relevant to this problem is given): * "mod_A": defines a package "mtest1" with an abstract public class "Class01A" that implements an interface "I01A", its "module-info.java" reads: ??????? "module mod_A { exports mtest1; }" * "mod_B": defines a package "mtest2" with a public "Class02A" which extends "mtest1.Class01A" and implements an interface "I02A" which extends "mtest1.I01A", its "module-info.java" reads: ??????? "module mod_B {? requires mod_A; ??? exports mtest2 to mod_C;? } * "mod_C": defines a package "mtest2" with a public "Class03A" which extends "mtest2.Class02A", its "module-info.java" reads: "module mod_C { exports mtest3; requires mod_B; }" The code doing the reflection resides in the unnamed module for the time being (it eventually will be part of a module). Running the script code is done against the following Java settings: -cp "%CLASSPATH%" --module-path F:\java9modules\out --add-modules mod_A,mod_B,mod_C In the first round reflecting Fields is used as a testbed. The reflection code at this stage is able to successfully skip over the closed "mod_B" module and arrive at "mod_A" classes. However, reflecting for "Class03A" instance is not able to access the defined *protected* static field "myClassName" in the superclass "Class01A" (the String value of that static field is: "class-mtest1.Class01A")! The debug output with the trailing stack trace for the runtime error is: about to load class [mtest3.Class03A] ???????? loaded, clz~toString: [class mtest3.Class03A] ???????? [java.lang.Class at 4973813a] package: [package mtest3] module: [module mod_C] org.rexxla.bsf.engines.rexx.RexxReflectJava9 at 6fb0d3ed -> reflect(rru): org.rexxla.bsf.engines.rexx.RexxReflectUtil at 16f7c8c1, field values: --->??? rajo????????? =[org.rexxla.bsf.engines.rexx.RexxAndJava at 24a35978] ??????? invocationType=[GET_FIELD_VALUE] ??????? reflectionType=[REFLECT_FIELD] ??????? bStrict?????? =[false] ??????? beanName????? =[mtest3.Class03A at 1563da5] ??????? bean????????? =[mtest3.Class03A at 1563da5] instanceof Class? [false] ??????? beanClz?????? =[class mtest3.Class03A] ??????? memberName??? =[MYCLASSNAME] ??????? rexxArgs[]??? =[[Ljava.lang.String;@df27fae], rexxArgs.length?? =[3]: [GETFIELDVALUE, mtest3.Class03A at 1563da5, MYCLASSNAME] ??????? tmpRexxArgs[] =[[Ljava.lang.String;@704921a5], tmpRexxArgs.length=[0]: [] ??????? funcArgs[]??? =[[Ljava.lang.Object;@727803de], funcArgs.length?? =[0]: [] ??????? bReturnJSO??? =[false] ??????? bTryCoercions =[true] <--- \\// RexxReflectJava9.reflectField: (1) in tmpClz.getSuperclass() loop: tmpClz=[mtest2.Class02A], BEFORE isExported() \\// RexxReflectJava9.reflectField: package of tmpClz not EXPORTED, hence SKIPPING tmpClz: [mtest2.Class02A] \\// RexxReflectJava9.reflectField: now checking?????????????????????????????? ---> tmpClz=[mtest1.Class01A] \\// RexxReflectJava9.reflectField: (1) in tmpClz.getSuperclass() loop: tmpClz=[mtest1.Class01A], BEFORE isExported() //\\ RexxReflectJava9.reflectField: (2) in tmpClz.getSuperclass() loop: tmpClz=[mtest1.Class01A], AFTER? isExported() RexxReflectJava9.processField(), arrived: -> [GET_FIELD_VALUE], tmpField=[protected static java.lang.String mtest1.Class01A.myClassName]: field=[MYCLASSNAME] in object=[mtest3.Class03A at 1563da5] RexxReflectJava9.processField(): => [GET_FIELD_VALUE]: found field=[MYCLASSNAME] in object=[mtest3.Class03A at 1563da5/mtest3.Class03A at 1563da5] oops GET-operation: tmpField "myClassName" caused exception "java.lang.IllegalAccessException: class org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access a member of class mtest1.Class01A (in module mod_A) with modifiers "protected static"" java.lang.reflect.InaccessibleObjectException: Unable to make field protected static java.lang.String mtest1.Class01A.myClassName accessible: module mod_A does not "opens? mtest1" to unnamed module @16022d9d ??????? at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) ??????? at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) ??????? at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source) ??????? at java.base/java.lang.reflect.Field.setAccessible(Unknown Source) ??????? at org.rexxla.bsf.engines.rexx.RexxReflectJava9.processField(RexxReflectJava9.java:294) ??????? at org.rexxla.bsf.engines.rexx.RexxReflectJava9.reflectField(RexxReflectJava9.java:113) ??????? at org.rexxla.bsf.engines.rexx.RexxReflectJava9.reflect(RexxReflectJava9.java:59) ??????? at org.rexxla.bsf.engines.rexx.RexxAndJava.javaCallBSF(RexxAndJava.java:3247) ??????? at org.rexxla.bsf.engines.rexx.RexxAndJava.javaCallBSF(RexxAndJava.java:4163) ??????? at org.rexxla.bsf.engines.rexx.RexxAndJava.jniRexxRunProgram(Native Method) ??????? at org.rexxla.bsf.engines.rexx.RexxEngine.apply(RexxEngine.java:1153) ??????? at org.rexxla.bsf.RexxDispatcher.main(RexxDispatcher.java:158) Doing the comparable operation - accessing a field named "myClassName" in Java code from "mtest3.Class03a" (in the main method) succeeds! Here is the output of running "mtest3.Class3A" for comparison: F:\java9modules>java --module-path out -m mod_C/mtest3.Class03A class mtest3.Class03A.main() ... ?????? getMyClassNameStatic()=[class-mtest1.Class01A] ?????? myClassName?????????? =[class-mtest1.Class01A] ?????? getMyName1()????????? =[from Class01A (static)] ???? o.getMyName2()????????? =[from Class02A (instance)] ???? o.myClassName?????????? =[class-mtest1.Class01A] So the question is, how can I reflectively access "mtest1.Class01A" static protected field "myClassName" from "mtest3.Class03a" in Java 9? The Java code for "mtest1.Class01A" (in "mod_A"), "mtest2.Class02A" (in "mod_B") and "mtest3.Class03A" (in "mod_C") is given below. ---rony "mod_A": package mtest1; abstract public class Class01A implements I01A { ??? protected static String myClassName = "class-mtest1.Class01A"; ??? protected static String myName1 = "from Class01A (static)"; ? ? protected String myName2 = "from Class01A (instance)"; ???? public static void main (String args[]) { ???? ??? System.err.println(Class01A.class+".main() ..."); ???? } ???? public static String getMyClassNameStatic() // static method in interface ???? { ???? ???? return myClassName; ???? } ???? abstract public String getMyClassName() ; // default method in interface ???? static protected String getMyName1() ???? { ???? ???? return myName1; ???? } ???? abstract protected String getMyName2(); } "mod_B": package mtest2; public class Class02A extends mtest1.Class01A implements I02A { ???? public static String myName1 = "from Class02A (static)"; ???? public String myName2 = "from Class02A (instance)"; public static void main (String args[]) { ???? ???? System.err.println(Class02A.class+".main() ..."); ???? ???? System.err.println(" getMyClassNameStatic()=["+ getMyClassNameStatic()+"]"); ???? ???? System.err.println(" getMyName1() =["+ getMyName1() +"]"); ???? ???? Class02A o=new Class02A(); ???? ???? System.err.println(" o.getMyName2() =["+o.getMyName2() +"]"); ???? ???? System.err.println(" o.getMyClassName() =["+o.getMyClassName() +"]"); ???? } ???? public String getMyClassName() ???? { ???? ???? return myClassName; ???? } ???? protected String getMyName2() ???? { ???? ???? return myName2; ???? } } "mod_C": package mtest3; public class Class03A extends mtest2.Class02A { ??? public static void main (String args[]) { ??????? System.err.println(Class03A.class+".main() ..."); ??????? System.err.println("?????? getMyClassNameStatic()=["+? getMyClassNameStatic()+"]"); ??????? System.err.println("?????? myClassName?????????? =["+? myClassName?????????? +"]"); ??????? System.err.println("?????? getMyName1()????????? =["+? getMyName1()????????? +"]"); ??????? Class03A o=new Class03A(); ??????? System.err.println("???? o.getMyName2()????????? =["+o.getMyName2()????????? +"]"); ??????? System.err.println("???? o.myClassName?????????? =["+o.myClassName?????????? +"]"); ??? } } for completeness the Interface classes: "mod_A": ?package mtest1; public interface I01A { ??? static public String? getMyClassNameStatic()??? // static method in interface ??? { ??????? System.err.println("\t<<<(static public getMyClassNameStatic() method from ["+mtest1.I01A.class+"])>>>"); ??????? return "interface-mtest1.I01A"; // myClassName; ??? } ??? default public String getMyClassName()????????? // default method in interface ??? { ??????? System.err.println("\t<<<(default public getMyClassName() method from ["+mtest1.I01A.class+"])>>>"); ??????? return "interface-mtest1.I01A"; // myClassName; ??? } } "mod_B": package mtest2; public interface I02A extends mtest1.I01A { ??? static public String? getMyClassNameStatic()??? // static method in interface ??? { ??????? System.err.println("\t<<<(static public getMyClassNameStatic() method from ["+mtest2.I02A.class+"])>>>"); ??????? return "interface-mtest2.I02A"; // myClassName; ??? } ??? default public String getMyClassName()????????? // default method in interface ??? { ??????? System.err.println("\t<<<(default public getMyClassName() method from ["+mtest2.I02A.class+"])>>>"); ??????? return "interface-mtest2.I02A"; // myClassName; ??? } } From jeffrey_kutcher at yahoo.com Mon Jan 15 21:50:57 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Mon, 15 Jan 2018 21:50:57 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <8813d25e-4bdb-8f90-0f8a-2982921c45cc@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <8813d25e-4bdb-8f90-0f8a-2982921c45cc@oracle.com> Message-ID: <1282610044.3739139.1516053057273@mail.yahoo.com> So developers have to be concerned with how the underlying classes are implemented and code to the implementation. This breaks the contract that is fundamental to Java. For example, the point to garbage collection was don't worry about how garbage collection works. Write your code in the pure sense and over time the garbage collector will get better due to the wizards that are interested in making the garbage collector scream. Leverage the technology and your code will inherently get better because the underlying technology being used is so much better. That holds true for the compiler and every other component of the JDK. Stay away from native functions. You will break portability. Again, there should be no concern about the underlying architecture. The example code provided in this post is 100% portable. So either reflection needs to be rewritten or deprecated or classes like Map and Box that can be reflected on needs rewriting so reflection can work properly. If a developer has to worry about how libraries are written, the libraries need to be rewritten. Maybe those libraries that have internal private classes need to provide the necessary getters and setters so that it won't be necessary to access those internal private classes. Instead of calling vbox.getChildren().add(..) maybe provide vbox.add(..) which will intern call?vbox.getChildren().add(..). Some libraries work as expected. ArrayList, BitSet or JRadioButton are good examples, among many others (proof it has be done). Just because you can doesn't mean you should. As a side note which may contribute to resolving this post's issue, internal classes should never be used. First, reflecting on a private class can possibly happen resulting in this post's issue. Second, if you're using an internal class then you're saying that the code will have no use elsewhere which contradicts write once, run anywhere because everyone knows that the point of objects is reuse and if you build something that good, it's a sin to keep it private. Other's will inevitably cut and paste the code elsewhere duplicating the code adding system bloat and now support in more than one location suggesting that the private class should be made public and accessible to others which will contribute to resolving this post's original issue. The internal private class giving us this issue is no longer private. Just because you can use internal classes doesn't mean you should. From a purist standpoint, this is an easy fix. Just don't do it by convention. The result is all related problems have a probability of zero that they will ever exist. That's pretty good for adhering to simple convention. On Monday, January 15, 2018, 1:24:30 PM CST, Alan Bateman wrote: On 15/01/2018 18:27, jeffrey kutcher wrote: > I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. It's always been possible for getClass() to return a non-public class and for the Method invoke in the code example to fail with IllegalAccessException. You may have just got lucky in the past.? At some point then I would expect the JavaFX modules to not be open by default so maybe now is the time to fix the issues. You can run with `--illegal-access=warn` to help track down other code that may be accidentally trying to access members of JDK internal classes.? You can run with `--illegal-access=deny` to see how that code behaves when the classes in the JDK modules are not open for illegal access. -Alan From david.holmes at oracle.com Mon Jan 15 22:02:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jan 2018 08:02:34 +1000 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1152555021.3613809.1516040839387@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> Message-ID: <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> On 16/01/2018 4:27 AM, jeffrey kutcher wrote: > > Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be. > You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated. > Introspection of?vbox.getChildren().add(x) should be just as simple as calling?vbox.getChildren().add(x) itself, otherwise why have introspection at all? I recall a very similar discussion in the past. You have to start with (public) statically known types, not with the dynamic type of the instances involved. In essence you have to reflectively emulate the method resolution process that occurs with a direct call and which uses static type information. David ----- > Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system. > I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances. > Looking for an Oracle to glean insight (implications of a lock down) > I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work. > I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation. > It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever. > > I'm sure there are others. > On Monday, January 15, 2018, 8:44:46 AM CST, Jochen Theodorou wrote: > > > > Am 12.01.2018 um 22:49 schrieb Micha? Zegan: >> well you rather know what getChildren returns, so you know what the >> returned object is, like generally. > > but as soon as the method you want to call is not on the statically > declared type, nor within a set of known statically types you are having > a problem. There is no good way (good in terms of usability and > performance) to get an arbitrary valid method object without static > information. You are always forced to traverse the hierarchy to find the > most accessible type. You will have to check access... and then of > course there the two bad scenarios where invocation and search are not > the same class or actual caller and caller are not equal. Which leads to > a lengthy search using reflection to find a method, transfer it to a > method handle and then let the invoker do the actual call. > > bye Jochen > From Alan.Bateman at oracle.com Tue Jan 16 09:18:20 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 Jan 2018 09:18:20 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: Message-ID: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> On 15/01/2018 20:56, Rony G. Flatscher wrote: > : > > > So the question is, how can I reflectively access "mtest1.Class01A" static protected field > "myClassName" from "mtest3.Class03a" in Java 9? > The scenario in your mail is complicated but if I read it correctly then I would expect this code in mtest3.Class03A (mod_C) to work: Class.forName("mtest1.Class01A").getDeclaredField("myClassName").get(null); I can't tell if the code in the unnamed module extends mtest1.Class01A or not. If it does then the same code should work, if it's not a sub-class then you should see IllegalAccessException being thrown. Your traces show InaccessibleObjectException being thrown so I think you are attempting to call setAccessible(true) to suppress the access check. You can't do on that protected members when the package is not open for deep reflection. In the example, mod_A would need to `opens mtest1` to allow this. -Alan. From peter.levart at gmail.com Tue Jan 16 10:05:39 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 16 Jan 2018 11:05:39 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> Message-ID: <2dce85ca-b293-bd4f-cde2-8ac3774cf91f@gmail.com> On 01/15/2018 11:02 PM, David Holmes wrote: > I recall a very similar discussion in the past. You have to start with > (public) statically known types, not with the dynamic type of the > instances involved. In essence you have to reflectively emulate the > method resolution process that occurs with a direct call and which > uses static type information. > > David > ----- Exactly, and in the example given by Jeffrey: javafx.scene.control.Button button1 = ...; javafx.scene.layout.VBox vbox = ...; vbox.getChildren().add(button1); The right translation of the last line to reflective invocations would be this (skipping exception handling): Method getChildrenMethod = javafx.scene.layout.VBox.class.getMethod("getChildren"); Object children = getChildrenMethod.invoke(vbox); Method addMethod = getChildrenMethod.getReturnType().getMethod("add", javafx.scene.control.Button.class); addMethod.invoke(children, button1); You see, the compiler (javac) uses static type information (of the locals for example) to locate the correct method. In case of method chaining, it uses the declared return type of the previous method, etc... Regards, Peter From jeffrey_kutcher at yahoo.com Tue Jan 16 13:08:38 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Tue, 16 Jan 2018 13:08:38 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> Message-ID: <1677507611.4168582.1516108118496@mail.yahoo.com> Trying to find documentation for the method resolution process. Found this: https://stackoverflow.com/questions/6021109/java-runtime-method-resolution which leads to http://www.cowtowncoder.com/blog/archives/2010/12/entry_436.html Maybe this is an opportunity for someone to put together a class library that does the right thing all the time regarding the?method resolution process. Does the Java library already have this somewhere in it's source? If so, why not expose it making it easier for others so those interested don't have to reinvent the wheel every time. Is there official documentation explaining the method resolution process somewhere? On Monday, January 15, 2018, 4:02:44 PM CST, David Holmes wrote: On 16/01/2018 4:27 AM, jeffrey kutcher wrote: >? > Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be. > You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated. > Introspection of?vbox.getChildren().add(x) should be just as simple as calling?vbox.getChildren().add(x) itself, otherwise why have introspection at all? I recall a very similar discussion in the past. You have to start with (public) statically known types, not with the dynamic type of the instances involved. In essence you have to reflectively emulate the method resolution process that occurs with a direct call and which uses static type information. David ----- > Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system. > I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances. > Looking for an Oracle to glean insight (implications of a lock down) > I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work. > I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation. > It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever. > > I'm sure there are others. >? ? ? On Monday, January 15, 2018, 8:44:46 AM CST, Jochen Theodorou wrote: >? >? > > Am 12.01.2018 um 22:49 schrieb Micha? Zegan: >> well you rather know what getChildren returns, so you know what the >> returned object is, like generally. > > but as soon as the method you want to call is not on the statically > declared type, nor within a set of known statically types you are having > a problem. There is no good way (good in terms of usability and > performance) to get an arbitrary valid method object without static > information. You are always forced to traverse the hierarchy to find the > most accessible type. You will have to check access... and then of > course there the two bad scenarios where invocation and search are not > the same class or actual caller and caller are not equal. Which leads to > a lengthy search using reflection to find a method, transfer it to a > method handle and then let the invoker do the actual call. > > bye Jochen > From dalibor.topic at oracle.com Tue Jan 16 13:19:39 2018 From: dalibor.topic at oracle.com (dalibor topic) Date: Tue, 16 Jan 2018 14:19:39 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1677507611.4168582.1516108118496@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> Message-ID: On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official documentation explaining the method resolution process somewhere? I'd suggest taking a look at the Java Language specification and JVM specification for details. cheers, dalibor topic -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From Rony.Flatscher at wu.ac.at Tue Jan 16 13:50:46 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 16 Jan 2018 14:50:46 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> Message-ID: On 16.01.2018 10:18, Alan Bateman wrote: > On 15/01/2018 20:56, Rony G. Flatscher wrote: >> : >> >> So the question is, how can I reflectively access "mtest1.Class01A" static protected field >> "myClassName" from "mtest3.Class03a" in Java 9? >> > The scenario in your mail is complicated but if I read it correctly then I would expect this code > in mtest3.Class03A (mod_C) to work: > Class.forName("mtest1.Class01A").getDeclaredField("myClassName").get(null); The problem here is, that no code from any of the modules is doing the reflection. Rather the reflection comes from packages on the class path, hence from the unnamed module. > I can't tell if the code in the unnamed module extends mtest1.Class01A or not. If it does then the > same code should work, if it's not a sub-class then you should see IllegalAccessException being > thrown. The excercise here is as follows: * there are classes in "mod_A" which get extended in "mod_B" and once more extended in "mod_C".? * Hence "mod_B" requires "mod_A", and "mod_C" requires "mod_B", where "mod_B" exports to "mod_C" only * These are the two chains of class extensions, one (mtest1.Class01A) defines an abstract public class with protected members, one (mtest1.Class01B) defines a public class with public members, hence: o one chain is:? + in "mod_A" there is the class "abstract public class mtest1.Class01A", which contains a protected static field named "myClassName", + in "mod_B" there is the class "public class Class02A extends mtest1.Class01A", has no static field named "myClassName", + in "mod_C" there is the class "publi class Class03A extends mtest2.Class02A", has no static field named "myClassName. o another chain is: + in "mod_A" there is the class "abstract public class mtest1.Class01B", which contains a protected static field named "myClassName", + in "mod_B" there is the class "public class Class02B extends mtest1.Class01B", has no static field named "myClassName", + in "mod_C" there is the class "public class Class03B extends mtest2.Class02B", has no static field named "myClassName. Two scenarios, one works, one causes the "IllegalAccessException": * works: creating an instance of "mtest3.Class03B" reflectively and then getting the value of the public static field "myClassName" in "mtest1.Class01B", works reflectively! * "IllegalAccessException": creating an instance of "mtest3.Class03A" reflectively and then getting the value of the public static field "myClassName" in "mtest1.Class01A", causes the "IllegalAccessException"; o Interestingly, + the first error message is: "java.lang.IllegalAccessException: class org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access a member of class mtest1.Class01A (in module mod_A) with modifiers "protected static" + in the catch block then trying a "setAccessible(true)" yields another "IllegalAccessException" now with the following error message: "java.lang.reflect.InaccessibleObjectException: Unable to make field protected static java.lang.String mtest1.Class01A.myClassName accessible: module mod_A does not "opens" mtest1" to unnamed module @33b37288" > Your traces show InaccessibleObjectException being thrown so I think you are attempting to call > setAccessible(true) to suppress the access check. You can't do on that protected members when the > package is not open for deep reflection. In the example, mod_A would need to `opens mtest1` to > allow this. And that is exactly the point. "mtest1.Class01A" is a superclass of "mtest3.Class03A". Therefore the protected members in the superclass "mtest1.Class01A" are public for its subclasses like "mtest3.Class03A". This is a very basic and important property of the Java language. Therefore I assume that this is a bug in the current implementation of Java 9, but want to know for sure. --- I asked this very question ("are protected members in superclasses regarded to be public for subclasses") almost nine months ago and it was agreed that "protected" in superclasses become effectively "public" for subclasses in Java 9 as well, so it was clear that when adjusting the bridge to Java 9 everything would work as in the past for the users. [For the past 17 years I have always adhered to these Java semantics in the Rexx/ooRexx-Java bridge, users of the bridge have only been able to access public classes and public members and protected classes and protected members of superclasses. There has never been a need for any deep reflection (nor "desire" to do so) as package private and private members have to be respected to be "private".] ---rony From jeffrey_kutcher at yahoo.com Tue Jan 16 13:57:49 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Tue, 16 Jan 2018 13:57:49 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> Message-ID: <1761664711.4166054.1516111069246@mail.yahoo.com> There's nothing in the JLS by that name however, section 4.5?Parameterized Types, might be close to what I'm looking for. Even in a single inheritance system, resolving methods and types is a multi-dimensional process. I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]). On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic wrote: On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official documentation explaining the method resolution process somewhere? I'd suggest taking a look at the Java Language specification and JVM specification for details. cheers, dalibor topic -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From jeffrey_kutcher at yahoo.com Tue Jan 16 14:07:11 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Tue, 16 Jan 2018 14:07:11 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1916258777.4142536.1516109389822@mail.yahoo.com> Message-ID: <1640008058.4167365.1516111631510@mail.yahoo.com> Thanks Dalibor. On Tuesday, January 16, 2018, 7:45:55 AM CST, dalibor topic wrote: Try https://docs.oracle.com/javase/specs/jls/se9/html/jls-15.html#jls-15.12.4 for example. On 16.01.2018 14:29, jeffrey kutcher wrote: > Did that. There's nothing in there by that name. > > On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic > wrote: > > > > > On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official > > documentation explaining the method resolution process somewhere? > > I'd suggest taking a look at the Java Language specification and JVM > specification for details. > > cheers, > dalibor topic > -- > Dalibor Topic | Principal Product Manager > Phone: +494089091214 | Mobile: +491737185961 > > > ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 M?nchen > Registergericht: Amtsgericht M?nchen, HRA 95603 > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher > > Oracle is committed to developing > practices and products that help protect the environment -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From kevin.rushforth at oracle.com Tue Jan 16 14:23:40 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 16 Jan 2018 06:23:40 -0800 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1761664711.4166054.1516111069246@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> Message-ID: <5A5E0AEC.3080906@oracle.com> There are many good reasons for using non-public classes in the implementation of a class library, the main one being to keep implementation details from leaking into the public API. So while I "get" that it causes difficulties in your specific case, I don't agree that the solution is to avoid using non-public syb-types. -- Kevin jeffrey kutcher wrote: > There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for. > Even in a single inheritance system, resolving methods and types is a multi-dimensional process. > I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]). > On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic wrote: > > > > On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official > documentation explaining the method resolution process somewhere? > I'd suggest taking a look at the Java Language specification and JVM > specification for details. > > cheers, > dalibor topic > From jeffrey_kutcher at yahoo.com Tue Jan 16 14:42:49 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Tue, 16 Jan 2018 14:42:49 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <5A5E0AEC.3080906@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> Message-ID: <1571739910.4199720.1516113769074@mail.yahoo.com> Then in this case, shouldn't getChildren() not be permitted? You're exposing a non-public sub-type. On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth wrote: There are many good reasons for using non-public classes in the implementation of a class library, the main one being to keep implementation details from leaking into the public API. So while I "get" that it causes difficulties in your specific case, I don't agree that the solution is to avoid using non-public syb-types. -- Kevin jeffrey kutcher wrote: >? There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for. > Even in a single inheritance system, resolving methods and types is a multi-dimensional process. > I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]). >? ? On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic wrote:? >? >? > > On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official > documentation explaining the method resolution process somewhere? > I'd suggest taking a look at the Java Language specification and JVM > specification for details. > > cheers, > dalibor topic >? From Alan.Bateman at oracle.com Tue Jan 16 15:03:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 16 Jan 2018 15:03:29 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> Message-ID: <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> On 16/01/2018 13:50, Rony G. Flatscher wrote: > The excercise here is as follows: > > * there are classes in "mod_A" which get extended in "mod_B" and > once more extended in "mod_C". > * Hence "mod_B" requires "mod_A", and "mod_C" requires "mod_B", > where "mod_B" exports to "mod_C" only > * These are the two chains of class extensions, one > (mtest1.Class01A) defines an abstract public class with protected > members, one (mtest1.Class01B) defines a public class with public > members, hence: > o one chain is: > + in "mod_A" there is the class "abstract public class > mtest1.Class01A", which contains a protected static field > named "myClassName", > + in "mod_B" there is the class "public class Class02A > extends mtest1.Class01A", has no static field named > "myClassName", > + in "mod_C" there is the class "publi class Class03A > extends mtest2.Class02A", has no static field named > "myClassName. > o another chain is: > + in "mod_A" there is the class "abstract public class > mtest1.Class01B", which contains a protected static field > named "myClassName", > If I read the earlier text correctly then Class01B.myClassName is public. > + in "mod_B" there is the class "public class Class02B > extends mtest1.Class01B", has no static field named > "myClassName", > + in "mod_C" there is the class "public class Class03B > extends mtest2.Class02B", has no static field named > "myClassName. > > Two scenarios, one works, one causes the "IllegalAccessException": > > * works: creating an instance of "mtest3.Class03B" reflectively and > then getting the value of the public static field "myClassName" in > "mtest1.Class01B", works reflectively! > I assume the caller is org.rexxla.bsf.engines.rexx.RexxReflectJava9 on the class path. Module mod_A exports mtest1, mtest1.Class1B is public, code on the class path can reflect on Class1B's public members. > * "IllegalAccessException": creating an instance of > "mtest3.Class03A" reflectively and then getting the value of the > public static field "myClassName" in "mtest1.Class01A", causes the > "IllegalAccessException"; > I think you mean "protected static". > * > > > o Interestingly, > + the first error message is: > "java.lang.IllegalAccessException: class > org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access > a member of class mtest1.Class01A (in module mod_A) with > modifiers "protected static" > This looks correct, RexxReflectJava9 is not a sub-class of mtest1.Class01A so it can't expect to reflect on Class01A's protected members. > + in the catch block then trying a "setAccessible(true)" > yields another "IllegalAccessException" now with the > following error message: > "java.lang.reflect.InaccessibleObjectException: Unable to > make field protected static java.lang.String > mtest1.Class01A.myClassName accessible: module mod_A does > not "opens" mtest1" to unnamed module @33b37288" > The InaccessibleObjectException looks correct too. > And that is exactly the point. "mtest1.Class01A" is a superclass of > "mtest3.Class03A". Therefore the protected members in the superclass > "mtest1.Class01A" are public for its subclasses like "mtest3.Class03A". Do you have an issue here? If code is added to mod_C/mtest3.Class03A to reflect on protected members of mod_A/mtest1.Class01A then I would expect it to work. Furthermore, I would expect setAccessible(true) to succeed on protected static members too (not instance members of course). That said, the scenario that you've shown is code in RexxReflectJava9 doing the reflection. It's not in a sub-class of mtest1.Class01A so it should not have any access to its protected members. -Alan From kevin.rushforth at oracle.com Tue Jan 16 15:14:31 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 16 Jan 2018 07:14:31 -0800 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1571739910.4199720.1516113769074@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> <1571739910.4199720.1516113769074@mail.yahoo.com> Message-ID: <5A5E16D7.8010409@oracle.com> No, the fact that it returns a subtype is irrelevant to a program that uses it as an instance of the base class. The key here is to know what the formal declaration of the return type is, either statically or via reflection (e.g., using Method::getGenericReturnType). -- Kevin jeffrey kutcher wrote: > Then in this case, shouldn't getChildren() not be permitted? You're > exposing a non-public sub-type. > > On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth > wrote: > > > There are many good reasons for using non-public classes in the > implementation of a class library, the main one being to keep > implementation details from leaking into the public API. So while I > "get" that it causes difficulties in your specific case, I don't agree > that the solution is to avoid using non-public syb-types. > > -- Kevin > > > jeffrey kutcher wrote: > > There's nothing in the JLS by that name however, section 4.5 > Parameterized Types, might be close to what I'm looking for. > > Even in a single inheritance system, resolving methods and types is > a multi-dimensional process. > > I still say that by not allowing private inner classes, resolving > this issue would be much easier (or providing the access methods in > the parent class of the private internal class would work also ... > calling getChildren() really is a dependency on the underlying classes > implementation which I really don't understand why is declared private > and not accessible. There has to be a good reason otherwise it should > be changed or eliminated [get rid of inner classes]). > > On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic > > wrote: > > > > > > > > On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official > > documentation explaining the method resolution process somewhere? > > I'd suggest taking a look at the Java Language specification and JVM > > specification for details. > > > > cheers, > > dalibor topic > > From Rony.Flatscher at wu.ac.at Tue Jan 16 18:37:57 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 16 Jan 2018 19:37:57 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> Message-ID: On 16.01.2018 16:03, Alan Bateman wrote: > On 16/01/2018 13:50, Rony G. Flatscher wrote: >> The excercise here is as follows: >> >> * there are classes in "mod_A" which get extended in "mod_B" and once more extended in "mod_C".? >> * Hence "mod_B" requires "mod_A", and "mod_C" requires "mod_B", where "mod_B" exports to >> "mod_C" only >> * These are the two chains of class extensions, one (mtest1.Class01A) defines an abstract >> public class with protected members, one (mtest1.Class01B) defines a public class with public >> members, hence: >> o one chain is:? >> + in "mod_A" there is the class "abstract public class mtest1.Class01A", which contains >> a protected static field named "myClassName", >> + in "mod_B" there is the class "public class Class02A extends mtest1.Class01A", has no >> static field named "myClassName", >> + in "mod_C" there is the class "publi class Class03A extends mtest2.Class02A", has no >> static field named "myClassName. >> o another chain is: >> + in "mod_A" there is the class "abstract public class mtest1.Class01B", which contains >> a protected static field named "myClassName", >> > If I read the earlier text correctly then Class01B.myClassName is public. Yes. > >> + in "mod_B" there is the class "public class Class02B extends mtest1.Class01B", has no >> static field named "myClassName", >> + in "mod_C" there is the class "public class Class03B extends mtest2.Class02B", has no >> static field named "myClassName. >> >> Two scenarios, one works, one causes the "IllegalAccessException": >> >> * works: creating an instance of "mtest3.Class03B" reflectively and then getting the value of >> the public static field "myClassName" in "mtest1.Class01B", works reflectively! >> > I assume the caller is org.rexxla.bsf.engines.rexx.RexxReflectJava9 on the class path. Module > mod_A exports mtest1, mtest1.Class1B is public, code on the class path can reflect on Class1B's > public members. > > >> * "IllegalAccessException": creating an instance of "mtest3.Class03A" reflectively and then >> getting the value of the public static field "myClassName" in "mtest1.Class01A", causes the >> "IllegalAccessException"; >> > I think you mean "protected static". Yes, sorry. > >> * >> >> >> >> o Interestingly, >> + the first error message is: "java.lang.IllegalAccessException: class >> org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access a member of class >> mtest1.Class01A (in module mod_A) with modifiers "protected static" >> > This looks correct, RexxReflectJava9 is not a sub-class of mtest1.Class01A so it can't expect to > reflect on Class01A's protected members. > > >> + in the catch block then trying a "setAccessible(true)" yields another >> "IllegalAccessException" now with the following error message: >> "java.lang.reflect.InaccessibleObjectException: Unable to make field protected static >> java.lang.String mtest1.Class01A.myClassName accessible: module mod_A does not >> "opens" mtest1" to unnamed module @33b37288" >> > The InaccessibleObjectException looks correct too. > > >> And that is exactly the point. "mtest1.Class01A" is a superclass of "mtest3.Class03A". Therefore >> the protected members in the superclass "mtest1.Class01A" are public for its subclasses like >> "mtest3.Class03A". > Do you have an issue here? If code is added to mod_C/mtest3.Class03A to reflect on protected > members of mod_A/mtest1.Class01A then I would expect it to work. Furthermore, I would expect > setAccessible(true) to succeed on protected static members too (not instance members of course). > > That said, the scenario that you've shown is code in RexxReflectJava9 doing the reflection. It's > not in a sub-class of mtest1.Class01A so it should not have any access to its protected members. Well that is probably the core of the problem: who needs to be the subclass, the reflector or the object to be reflected upon (whose inheritance tree is being walked up towards the root class, and if public or protected members are found reflectively accessed)? Or conceptually: the "mtest3.Class03A object" is clearly from a subclass of "mtest1.Class01A". Why should it then not be allowed to access the public and protected members in the superclass "mtest1.Class01A" via reflection on behalf of the "mtest3.Class03A object", which gets supplied in the reflective access and which can be used to check whether the access would be legal or not? ---rony From john.r.rose at oracle.com Tue Jan 16 19:38:45 2018 From: john.r.rose at oracle.com (John Rose) Date: Tue, 16 Jan 2018 11:38:45 -0800 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> Message-ID: <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> On Jan 16, 2018, at 10:37 AM, Rony G. Flatscher wrote: > > Well that is probably the core of the problem: who needs to be the subclass, the reflector or the > object to be reflected upon (whose inheritance tree is being walked up towards the root class, and > if public or protected members are found reflectively accessed)? Quick comment: With the core reflection API, the reflector's permissions are derived from the reflector, not the class being investigated. This works fine when a class reflects itself but is not so good for frameworks. The reflection API in java.lang.invoke, though more limited, does support rights delegation through the Lookup object. This means that your "patient" class doesn't need to do the reflection work; it can merely create a private lookup in itself and hand it to the framework, which can do the reflection on behalf of the patient. You can even mix the old and new mechanisms (a point I don't think I've seen on this thread, although I haven't read it all). Given a private lookup object in some patient P, a framework class F can create a method handle on a core reflection API point that is @CallerSensitive (like Class.forName, etc. etc.) which allows F to call that API point with the same rights as P, not F. HTH ? John From blackdrag at gmx.org Tue Jan 16 21:08:28 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Tue, 16 Jan 2018 22:08:28 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> Message-ID: <100dacea-e4ad-2c61-b077-2083d4e1a11d@gmx.org> On 16.01.2018 20:38, John Rose wrote: > On Jan 16, 2018, at 10:37 AM, Rony G. Flatscher wrote: >> >> Well that is probably the core of the problem: who needs to be the subclass, the reflector or the >> object to be reflected upon (whose inheritance tree is being walked up towards the root class, and >> if public or protected members are found reflectively accessed)? > > Quick comment: > > With the core reflection API, the reflector's permissions > are derived from the reflector, not the class being investigated. > > This works fine when a class reflects itself but is not so good for frameworks. well said. > The reflection API in java.lang.invoke, though more limited, does > support rights delegation through the Lookup object. This means > that your "patient" class doesn't need to do the reflection work; > it can merely create a private lookup in itself and hand it to the > framework, which can do the reflection on behalf of the patient. which is a big deal of a problem if the class, that is supposed to be exposed to the framework is not under the control of the framework. And there is no good solution (rewriting the bytecode for example is no good solution) > You can even mix the old and new mechanisms (a point I don't > think I've seen on this thread, although I haven't read it all). > Given a private lookup object in some patient P, a framework > class F can create a method handle on a core reflection API > point that is @CallerSensitive (like Class.forName, etc. etc.) > which allows F to call that API point with the same rights as > P, not F. Groovy is using that kind of logic in the initial invokedynamic implementation. The problem is that Reflection allowed much more. Frameworks (and whole languages) did grow around these capabilities. In Java9 they have no alternative implementation anymore. And I'd like to especially mention @CallerSensitive... This makes it very very difficult for frameworks. The more general, the more you have to work around that kind of logic - up to the point where no workaround is possible anymore (short of opening for deep reflection, living with thousands of warnings and an in acceptable commandline usage). bye Jochen From john.r.rose at oracle.com Tue Jan 16 21:34:27 2018 From: john.r.rose at oracle.com (John Rose) Date: Tue, 16 Jan 2018 13:34:27 -0800 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <100dacea-e4ad-2c61-b077-2083d4e1a11d@gmx.org> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <100dacea-e4ad-2c61-b077-2083d4e1a11d@gmx.org> Message-ID: <013CBABD-5CD0-4180-8072-B538FD8113CD@oracle.com> On Jan 16, 2018, at 1:08 PM, Jochen Theodorou wrote: > > which is a big deal of a problem if the class, that is supposed to be exposed to the framework is not under the control of the framework. And there is no good solution (rewriting the bytecode for example is no good solution) Yep. We didn't solve all those problems in 9 since they are very complicated trade-offs, and the "new kid on the block" of enforced encapsulation is taking ground previously occupied by the "neighborhood gang" of legitimate frameworks. So we need a way to give those frameworks what they need, in a way that still make encapsulation preservable and checkable. (My mental model for this tradeoff is, how can I AOT-compile as much as possible, while still leaving open some legitimate hooks for frameworks to operate? I.e., enabling AOT and framework intercession at the same time is what winning looks like.) The privateLookup API will allow a framework to get full-power "guest" status in an arbitrary uncooperative class. At least, that's the advertisement. Will that help, or is there some bad fine-print to that API? From david.holmes at oracle.com Tue Jan 16 22:14:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Jan 2018 08:14:35 +1000 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1677507611.4168582.1516108118496@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> Message-ID: <11301ce9-c3cd-57d9-d6e4-1cbb2bf1ead7@oracle.com> On 16/01/2018 11:08 PM, jeffrey kutcher wrote: > Trying to find documentation for the method resolution process. The JVMS defines the rules for method resolution (and subsequent selection). David ------ > Found this: > > https://stackoverflow.com/questions/6021109/java-runtime-method-resolution > > which leads to > > http://www.cowtowncoder.com/blog/archives/2010/12/entry_436.html > > Maybe this is an opportunity for someone to put together a class library that does the right thing all the time regarding the?method resolution process. Does the Java library already have this somewhere in it's source? If so, why not expose it making it easier for others so those interested don't have to reinvent the wheel every time. > > > Is there official documentation explaining the method resolution process somewhere? > > > On Monday, January 15, 2018, 4:02:44 PM CST, David Holmes wrote: > > > > > > On 16/01/2018 4:27 AM, jeffrey kutcher wrote: >> >> Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be. >> You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated. >> Introspection of?vbox.getChildren().add(x) should be just as simple as calling?vbox.getChildren().add(x) itself, otherwise why have introspection at all? > > I recall a very similar discussion in the past. You have to start with > (public) statically known types, not with the dynamic type of the > instances involved. In essence you have to reflectively emulate the > method resolution process that occurs with a direct call and which uses > static type information. > > David > > ----- > >> Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system. >> I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances. >> Looking for an Oracle to glean insight (implications of a lock down) >> I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work. >> I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation. >> It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever. >> >> I'm sure there are others. >> ? ? ? On Monday, January 15, 2018, 8:44:46 AM CST, Jochen Theodorou wrote: >> >> >> >> Am 12.01.2018 um 22:49 schrieb Micha? Zegan: >>> well you rather know what getChildren returns, so you know what the >>> returned object is, like generally. >> >> but as soon as the method you want to call is not on the statically >> declared type, nor within a set of known statically types you are having >> a problem. There is no good way (good in terms of usability and >> performance) to get an arbitrary valid method object without static >> information. You are always forced to traverse the hierarchy to find the >> most accessible type. You will have to check access... and then of >> course there the two bad scenarios where invocation and search are not >> the same class or actual caller and caller are not equal. Which leads to >> a lengthy search using reflection to find a method, transfer it to a >> method handle and then let the invoker do the actual call. >> >> bye Jochen >> From Rony.Flatscher at wu.ac.at Wed Jan 17 12:18:42 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 17 Jan 2018 13:18:42 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> Message-ID: <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> On 16.01.2018 20:38, John Rose wrote: > On Jan 16, 2018, at 10:37 AM, Rony G. Flatscher > wrote: >> >> Well that is probably the core of the problem: who needs to be the subclass, the reflector or the >> object to be reflected upon (whose inheritance tree is being walked up towards the root class, and >> if public or protected members are found reflectively accessed)? > > Quick comment: > > With the core reflection API, the reflector's permissions > are derived from the reflector, not the class being investigated. > > This works fine when a class reflects itself but is not so good for frameworks. > > The reflection API in java.lang.invoke, though more limited, does > support rights delegation through the Lookup object. ?This means > that your "patient" class doesn't need to do the reflection work; > it can merely create a private lookup in itself and hand it to the > framework, which can do the reflection on behalf of the patient. > > You can even mix the old and new mechanisms (a point I don't > think I've seen on this thread, although I haven't read it all). > Given a private lookup object in some patient P, a framework > class F can create a method handle on a core reflection API > point that is @CallerSensitive (like Class.forName, etc. etc.) > which allows F to call that API point with the same rights as > P, not F. Thank you for the pointer, reading through it seems that the access problem cannot be solved with that package either. The problem at hand is the fact that the framework in question ("BSF4ooRexx", a bridge between ooRexx implemented in C++ and Java) is *not* part of any class hierarchy in the module(s) nor related to any specific module. A simple example ooRexx program using a Java class that gets intertwined with the three test modules mod_A (exports mtest1), mod_B (requires mod_A, exports mtest2 to mod_C), mod_C (exports mtest3): o=.bsf~new("mtest3.Class03A")???????????? -- create Java object, get and assign proxy ooRexx object say "o:" o "o~myClassName:" o~myClassName -- get (static) field value in "mtest1.Class01A", accessible via inheritance rules ::requires BSF.CLS?? -- direct interpreter to load the Java bridge The basic logic is as follows: * load the class "mtest3.Class03A", reflectively get a constructor and create a Java object, store it in a Java repository return the key to allow looking up the object * ooRexx creates an ooRexx proxy object which will forward all unknown messages to the bridge, which in turn determines what is sought and uses reflection to carry out the desired operation * the second statement shows the ooRexx object name (the key into the Java registry) and uses the bridge to have it look up the field named "myClassName" (there is no message by that name on the ooRexx side, so the bridge is used to lookup the proxied Java object; a message is indicated by the message operator '~', the tilde, where the receiver is placed left of it and the message name right of it) The reflection logic in the bridge is (simplified) as follows: * use the object's 'o' class and look up its declared methods or fields, analyze further, if member is 'public', otherwise o iterate over all of its superclasses looking up in each class all declared methods or fields, analyze further, if member is 'public' *or* 'protected' (looking up a superclass 'protected' is regarded to be 'public' for the subclass) and, if a matching member is found, carry out the operation (in this case a Field.get()) and return its value This way it is assured that the users of the bridge are never able to get at private or package private members (nor that they are able to invoke a protected member in the object's type itself). This logic has been working since Java 1.1 (sic!) throughout all Java versions using Java's reflection infrastructure. Java 9 currently breaks this. Is there currently (in the current implementation) any other means to achieve the same, mandatory needed functionality in Java 9 that I could use? --- The current implementation seems to do the following: it checks whether a class is from an? exported package, if not it creates an exception. If the class is exported it then checks whether the reflected member is public, if not it creates an exception. What I would have expected is this: everything like the current implementation, but in the second step, if the member is not public, check whether the object's class for which the reflection takes place is a subclass of the class that is being checked and if so, allow protected members to be accessed as well. Hence: in the example presented, the packages 'mtest1' and 'mtest3' are exported, the class 'mtest3.Class03A' extends 'mtest2.Class02A' which extends 'mtest1.Class01A'. Therefore the object of type 'mtest3.Class03A' is a subclass of 'mtest1.Class01A', hence as 'mtest1' is exported all 'mtest1.Class01A' public *and* protected members should be accessible to the object of the subtype 'mtest3.Class03A' via reflection in Java 9 as well. ---rony From Alan.Bateman at oracle.com Wed Jan 17 13:12:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jan 2018 13:12:41 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> Message-ID: <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> On 17/01/2018 12:18, Rony G. Flatscher wrote: > : > > The reflection logic in the bridge is (simplified) as follows: > > * use the object's 'o' class and look up its declared methods or > fields, analyze further, if member is 'public', otherwise > > o iterate over all of its superclasses looking up in each class > all declared methods or fields, analyze further, if member is > 'public' *or* 'protected' (looking up a superclass 'protected' > is regarded to be 'public' for the subclass) and, if a > matching member is found, carry out the operation (in this > case a Field.get()) and return its value > > This way it is assured that the users of the bridge are never able to > get at private or package private members (nor that they are able to > invoke a protected member in the object's type itself). > Accessibility has been significantly upgraded in Java SE 9. "public" used to imply accessible to everyone, now we have "public to everyone", "public to friend modules" or "public within a module". If you want the above logic to work with modules then it will need to be updated to handle classes in named modules. It will essentially amount to overhauling the above to find the accessible members rather the public members (in public classes). BTW: On protected members then I assume they have never been accessible to the bridge, maybe the bridge is using setAccessible to suppress the access checks for those cases? > > --- > > The current implementation seems to do the following: it checks > whether a class is from an? exported package, if not it creates an > exception. If the class is exported it then checks whether the > reflected member is public, if not it creates an exception. > > What I would have expected is this: everything like the current > implementation, but in the second step, if the member is not public, > check whether the object's class for which the reflection takes place > is a subclass of the class that is being checked and if so, allow > protected members to be accessed as well. Hence: in the example > presented, the packages 'mtest1' and 'mtest3' are exported, the class > 'mtest3.Class03A' extends 'mtest2.Class02A' which extends > 'mtest1.Class01A'. Therefore the object of type 'mtest3.Class03A' is a > subclass of 'mtest1.Class01A', hence as 'mtest1' is exported all > 'mtest1.Class01A' public *and* protected members should be accessible > to the object of the subtype 'mtest3.Class03A' via reflection in Java > 9 as well. You should find that the protected members are accessible in the sub-class (mtest3.Class03A in this case). They won't be accessible to the bridge code of course (as least not without some kind of rights delegation as John mentioned). -Alan From jeffrey_kutcher at yahoo.com Wed Jan 17 15:10:57 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Wed, 17 Jan 2018 15:10:57 +0000 (UTC) Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> Message-ID: <133355072.5035437.1516201857454@mail.yahoo.com> Disclaimer: I don't have access to the full picture but wanted to throw this out there. Maybe it will be useful. Maybe you've already considered doing something like this and shot it down a long time ago.?I don't know if there are show stoppers and I don't have the end goal in mind. However it seems like it might be a useful way to simplify what I've read so far. The following suggestion?might untangle the web which appears to be growing though it may be too late to introduce. Suggestion: A closure might provide the plumbing you're looking for rather than using public/private/protected modifiers which seems to be gaining overloaded status. public/private/protected modifiers?should go back to their original meaning and kept simple. A closure would allow classes by default, no or default?access/authority/permission/security?unless being owned by a container in which case would assume it's owner's access. This might eliminate the expanding access issues your finding and be more robust in the process, maybe even collapse the code. You'll have to provide an environment to accomplish this. If a class "has a" or "is a" sub-class, the sub-class would take on the access of the parent class until such time as it deferred that access for more specific purposes. If not, any "has a" or "is a" sub-class would assume the role of its owner. "has a" and "is a" sub-classes would have to reference their owner's environment to gain access permission otherwise no (or default) access would be granted. Coming up with the right convention should prove to be elegant and introduce little to no overhead. On Wednesday, January 17, 2018, 7:14:23 AM CST, Alan Bateman wrote: On 17/01/2018 12:18, Rony G. Flatscher wrote: > : > > The reflection logic in the bridge is (simplified) as follows: > >? * use the object's 'o' class and look up its declared methods or >? ? fields, analyze further, if member is 'public', otherwise > >? ? ? o iterate over all of its superclasses looking up in each class >? ? ? ? all declared methods or fields, analyze further, if member is >? ? ? ? 'public' *or* 'protected' (looking up a superclass 'protected' >? ? ? ? is regarded to be 'public' for the subclass) and, if a >? ? ? ? matching member is found, carry out the operation (in this >? ? ? ? case a Field.get()) and return its value > > This way it is assured that the users of the bridge are never able to > get at private or package private members (nor that they are able to > invoke a protected member in the object's type itself). > Accessibility has been significantly upgraded in Java SE 9. "public" used to imply accessible to everyone, now we have "public to everyone", "public to friend modules" or "public within a module". If you want the above logic to work with modules then it will need to be updated to handle classes in named modules. It will essentially amount to overhauling the above to find the accessible members rather the public members (in public classes). BTW: On protected members then I assume they have never been accessible to the bridge, maybe the bridge is using setAccessible to suppress the access checks for those cases? > > --- > > The current implementation seems to do the following: it checks > whether a class is from an? exported package, if not it creates an > exception. If the class is exported it then checks whether the > reflected member is public, if not it creates an exception. > > What I would have expected is this: everything like the current > implementation, but in the second step, if the member is not public, > check whether the object's class for which the reflection takes place > is a subclass of the class that is being checked and if so, allow > protected members to be accessed as well. Hence: in the example > presented, the packages 'mtest1' and 'mtest3' are exported, the class > 'mtest3.Class03A' extends 'mtest2.Class02A' which extends > 'mtest1.Class01A'. Therefore the object of type 'mtest3.Class03A' is a > subclass of 'mtest1.Class01A', hence as 'mtest1' is exported all > 'mtest1.Class01A' public *and* protected members should be accessible > to the object of the subtype 'mtest3.Class03A' via reflection in Java > 9 as well. You should find that the protected members are accessible in the sub-class (mtest3.Class03A in this case). They won't be accessible to the bridge code of course (as least not without some kind of rights delegation as John mentioned). -Alan From Rony.Flatscher at wu.ac.at Wed Jan 17 18:53:10 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 17 Jan 2018 19:53:10 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> Message-ID: On 17.01.2018 14:12, Alan Bateman wrote: > On 17/01/2018 12:18, Rony G. Flatscher wrote: >> : >> >> The reflection logic in the bridge is (simplified) as follows: >> >> * use the object's 'o' class and look up its declared methods or fields, analyze further, if >> member is 'public', otherwise >> >> o iterate over all of its superclasses looking up in each class all declared methods or >> fields, analyze further, if member is 'public' *or* 'protected' (looking up a superclass >> 'protected' is regarded to be 'public' for the subclass) and, if a matching member is >> found, carry out the operation (in this case a Field.get()) and return its value >> >> This way it is assured that the users of the bridge are never able to get at private or package >> private members (nor that they are able to invoke a protected member in the object's type itself). >> > Accessibility has been significantly upgraded in Java SE 9. "public" used to imply accessible to > everyone, now we have "public to everyone", "public to friend modules" or "public within a > module".? If you want the above logic to work with modules then it will need to be updated to > handle classes in named modules. It will essentially amount to overhauling the above to find the > accessible members rather the public members (in public classes). Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a module, but having a need to access public types from exported packages and get reflective access to objects supertype's protected members? This actually is an urgent need for my bridge. The BSF4ooRexx package comes with a wealth of samples to show ooRexx programmers how easy it is to use Java as a huge external ooRexx class library. Some of these samples in Java 9 now cause warnings, e.g. one example: ... cut ... WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.rexxla.bsf.engines.rexx.RexxAndJava (file:/C:/Program%20Files%20(x86)/BSF4ooRexx/bsf4ooRexx-v600-20180101-bin.jar) to method sun.java2d.SunGraphics2D.setRenderingHint(java.awt.RenderingHints$Key,java.lang.Object) WARNING: Please consider reporting this to the maintainers of org.rexxla.bsf.engines.rexx.RexxAndJava WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release ... cut ... So all of a sudden the reflection does not work anymore. The reason being that the graphic "g" context in javax.swing.JComponent's paintComponent (java.awt.Graphics g) is actually a "sun.java2d.SunGraphics2D" object, that according to this Java 9 warning is not exported and will be an error in Java 10. (I have assumed, that if I would use a reflection Method object from javax.swing.JComponent (which is protected and can only be accessed by ooRexx, if a dynamically created subclass of JComponent is created from which the work is done) for setRenderingHint() that invoking the Method object with the Sun object would succeed.) There are other use-cases where such warnings occur. No user of the ooRexx-Java bridge knows anything of the new module system in Java 9, nor should she have a need to know if these sample programs have worked flawlessly in the past. Therefore I am seriously interested to find a solution that adheres to the Java 9 module system. > BTW: On protected members then I assume they have never been accessible to the bridge, maybe the > bridge is using setAccessible to suppress the access checks for those cases? Yes. (Have been hoping that that need would be forgone eventually, when that reflection area gets reworked, updated for this protected member case, if the object to be worked on stems from a subclass to allow exactly the same thing the Java compiler has allowed for, this time for an interpreted language.) > >> >> --- >> >> The current implementation seems to do the following: it checks whether a class is from an? >> exported package, if not it creates an exception. If the class is exported it then checks whether >> the reflected member is public, if not it creates an exception. >> >> What I would have expected is this: everything like the current implementation, but in the second >> step, if the member is not public, check whether the object's class for which the reflection >> takes place is a subclass of the class that is being checked and if so, allow protected members >> to be accessed as well. Hence: in the example presented, the packages 'mtest1' and 'mtest3' are >> exported, the class 'mtest3.Class03A' extends 'mtest2.Class02A' which extends 'mtest1.Class01A'. >> Therefore the object of type 'mtest3.Class03A' is a subclass of 'mtest1.Class01A', hence as >> 'mtest1' is exported all 'mtest1.Class01A' public *and* protected members should be accessible to >> the object of the subtype 'mtest3.Class03A' via reflection in Java 9 as well. > You should find that the protected members are accessible in the sub-class (mtest3.Class03A in > this case). They won't be accessible to the bridge code of course (as least not without some kind > of rights delegation as John mentioned). Not with Java 9 javac, the following Java program: public class TestUse { ??? public static void main (String args[]) { ??????? mtest3.Class03A o=new mtest3.Class03A(); ??????? System.out.println("o: "+o+", myClassName: "+o.myClassName); ??? } } yields the error message: F:\work\svn\bsf4oorexx\trunk\bsf4oorexx.dev\testUnits\bsf4rexx\java9modules>java? -cp ".;F:\work\svn\bsf4oorexx\trunk;.;C:\Program Files (x86)\BSF4ooRexx\bsf4ooRexx-v600-20180101-bin.jar;C:\Program Files (x86)\BSF4ooRexx\jni4net.j-0.8.8.0.jar;C:\Program Files (x86)\BSF4ooRexx\oorexx.net.jar;." --module-path out --add-modules mod_A,mod_B,mod_C TestUse Exception in thread "main" java.lang.IllegalAccessError: tried to access field mtest1.Class01A.myClassName from class TestUse ??????? at TestUse.main(TestUse.java:5) Note: the TestUse class will be part of the unnamed module. --- Ad "rights delegation mechanism": would that really allow the bridge from the unnamed module to do what is needed and therefore sought, ie. accessing protected members in exported superclasses? If so, how? Would you have any links for further reading? If you could tell me that I misunderstood the access rules of the java.lang.invoke package and that it was possible to use it and/or the jdk.dynalink package to overcome, I would definitely try to apply that to solve this issue, as this is really very important for the users of the bridge for whom it is simply impossible to solve this. --- Alternatively: what is the problem that you see for the Java 9 module system, if access checking is done with the extension of also allowing protected members to be used, if the object on which the reflective access is to be carried out is indeed from a subclass of the defining class of the java.lang.reflect object being supplied? ---rony From Alan.Bateman at oracle.com Wed Jan 17 19:43:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jan 2018 19:43:38 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> Message-ID: On 17/01/2018 18:53, Rony G. Flatscher wrote: > > : > > > Not with Java 9 javac, the following Java program: > > public class TestUse > { > ??? public static void main (String args[]) { > ??????? mtest3.Class03A o=new mtest3.Class03A(); > ??????? System.out.println("o: "+o+", myClassName: "+o.myClassName); > ??? } > } > > yields the error message: > > F:\work\svn\bsf4oorexx\trunk\bsf4oorexx.dev\testUnits\bsf4rexx\java9modules>java > -cp ".;F:\work\svn\bsf4oorexx\trunk;.;C:\Program Files > (x86)\BSF4ooRexx\bsf4ooRexx-v600-20180101-bin.jar;C:\Program Files > (x86)\BSF4ooRexx\jni4net.j-0.8.8.0.jar;C:\Program Files > (x86)\BSF4ooRexx\oorexx.net.jar;." --module-path out --add-modules > mod_A,mod_B,mod_C TestUse > Exception in thread "main" java.lang.IllegalAccessError: tried to > access field mtest1.Class01A.myClassName from class TestUse > ??????? at TestUse.main(TestUse.java:5) > If it compiles then it should run so something isn't right here. I just tried this with JDK 9 and JDK 10 EA builds but couldn't duplicate it (the compilation fails as expected because myClassName has protected access in mod_A/mtest1.Class01A). Would you mind zipping up the sources for mod_A, mod_B, and mod_C so we can duplicate this? -Alan From Alan.Bateman at oracle.com Thu Jan 18 09:58:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Jan 2018 09:58:38 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> Message-ID: <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> On 17/01/2018 18:53, Rony G. Flatscher wrote: > > : > > Would you have concrete suggestions for this use-case, i.e. a > framework that is not part of a module, but having a need to access > public types from exported packages and get reflective access to > objects supertype's protected members? I think it would be better to start with public members as protected is complicated (and hasn't changed with modules once you establish the class declaring the member is accessible). For your example, you've got a reference to a java.awt.Graphics2D object, the actual implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the superclass and interfaces to find a public class or interface that defines the target setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it were non public. Can you extend this check to test if the class is in a package exported by its module. For the example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export sun.java2d to everyone. Here is a code fragment to test this: Class clazz = graphicsObj.getClass(); boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). If you can incorporate this check into the bridge then I suspect you'll find most of the examples will work. -Alan From jeffrey_kutcher at yahoo.com Thu Jan 18 13:59:26 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Thu, 18 Jan 2018 13:59:26 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <5A5E0AEC.3080906@oracle.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> Message-ID: <1401341799.487077.1516283966017@mail.yahoo.com> In this example, if I can access and execute the method vbox.getChildren().add() with no warnings or errors, which is accessing a non-public class, then I should also able to access that very same class and execute the same method using reflection with no warnings or errors and expect the same results. If not, the inconsistency may be a bug, feature or broken. Maybe that's the way it's suppose to work. Inconsistency however tells me something isn't quite right and if ignored will cause more issues later on. In this particular case, a lock down will completely stop currently working code. I'd like to not see that happen. If it does? I'll move on. I don't disagree that there are good reasons for using non-public classes. On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth wrote: There are many good reasons for using non-public classes in the implementation of a class library, the main one being to keep implementation details from leaking into the public API. So while I "get" that it causes difficulties in your specific case, I don't agree that the solution is to avoid using non-public syb-types. -- Kevin jeffrey kutcher wrote: >? There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for. > Even in a single inheritance system, resolving methods and types is a multi-dimensional process. > I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]). >? ? On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic wrote:? >? >? > > On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official > documentation explaining the method resolution process somewhere? > I'd suggest taking a look at the Java Language specification and JVM > specification for details. > > cheers, > dalibor topic >? From blackdrag at gmx.org Thu Jan 18 14:34:26 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Thu, 18 Jan 2018 15:34:26 +0100 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1401341799.487077.1516283966017@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> <1401341799.487077.1516283966017@mail.yahoo.com> Message-ID: <806d80bb-74c5-c12b-b550-dd8008cbf219@gmx.org> Am 18.01.2018 um 14:59 schrieb jeffrey kutcher: > In this example, if I can access and execute the method vbox.getChildren().add() with no warnings or errors, which is accessing a non-public class In Java the static type is the important one. Of course you can do VBox vbox = .... // returns implementation from nonexported package vbox.getChildren().add() This will work as long as VBox is accessible. the getChildren call is not done using the implementation type, but VBox. And for this it does not matter that vbox has a "forbidden" type at runtime at all. Conclusively you have to do more at runtime if all you have is the runtime type of vbox. You have to check if this type is accessible to you and if not go to a super class. You would then maybe also end up in Vbox and invoke the method you get for VBox, not for the implementation of VBox. You still do a virtual method call, so the implementation method is still called > then I should also able to access that very same class and execute the same method using reflection with no warnings or errors and expect the same results. If not, the inconsistency may be a bug, feature or broken. design choice I would claim. > Maybe that's the way it's suppose to work. Inconsistency however tells me something isn't quite right and if ignored will cause more issues later on. That is because Reflection has been declared as broken in that it has a tradition of allowing too much. Now this has been restricted, leading to problems. But in this case you just have to use the type the Java compiler would, which is VBOX. > In this particular case, a lock down will completely stop currently working code. You are not alone here. bye Jochen From Rony.Flatscher at wu.ac.at Thu Jan 18 15:07:09 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Thu, 18 Jan 2018 16:07:09 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> Message-ID: <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> Dear Alan: tried to come up with a "cleaner" version to zip it up, however the error would not occur there. --- As I have to get off for today, I just zipped up yesterday's version "as is" that exhibits that compile error and attach it to this e-mail. Just unzip it, go into "java9module" (this is on Windows): * ./ contains the (Windows) batch files and TestUse.java o to compile all sources: + 1_compile.cmd o to compile TestUse.java: + 5_compile_and_run_TestUse.cmd * src/ contains the sources * out/ contains the compiled stuff If you prefer and can wait until next week, I will try to find out the differences between the "cleaned" version (removed all Interfaces for that version) and yesterday's version and report back. ---rony On 17.01.2018 20:43, Alan Bateman wrote: > On 17/01/2018 18:53, Rony G. Flatscher wrote: >> >> : >> >> >> Not with Java 9 javac, the following Java program: >> >> public class TestUse >> { >> ??? public static void main (String args[]) { >> ??????? mtest3.Class03A o=new mtest3.Class03A(); >> ??????? System.out.println("o: "+o+", myClassName: "+o.myClassName); >> ??? } >> } >> >> yields the error message: >> >> F:\work\svn\bsf4oorexx\trunk\bsf4oorexx.dev\testUnits\bsf4rexx\java9modules>java? -cp >> ".;F:\work\svn\bsf4oorexx\trunk;.;C:\Program Files >> (x86)\BSF4ooRexx\bsf4ooRexx-v600-20180101-bin.jar;C:\Program Files >> (x86)\BSF4ooRexx\jni4net.j-0.8.8.0.jar;C:\Program Files (x86)\BSF4ooRexx\oorexx.net.jar;." >> --module-path out --add-modules mod_A,mod_B,mod_C TestUse >> Exception in thread "main" java.lang.IllegalAccessError: tried to access field >> mtest1.Class01A.myClassName from class TestUse >> ??????? at TestUse.main(TestUse.java:5) >> > If it compiles then it should run so something isn't right here. I just tried this with JDK 9 and > JDK 10 EA builds but couldn't duplicate it (the compilation fails as expected because myClassName > has protected access in mod_A/mtest1.Class01A). > > Would you mind zipping up the sources for mod_A, mod_B, and mod_C so we can duplicate this? > > -Alan From Rony.Flatscher at wu.ac.at Thu Jan 18 15:11:15 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Thu, 18 Jan 2018 16:11:15 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> Message-ID: On 18.01.2018 10:58, Alan Bateman wrote: > On 17/01/2018 18:53, Rony G. Flatscher wrote: >> >> : >> >> Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a >> module, but having a need to access public types from exported packages and get reflective access >> to objects supertype's protected members? > I think it would be better to start with public members as protected is complicated (and hasn't > changed with modules once you establish the class declaring the member is accessible). > > For your example, you've got a reference to a java.awt.Graphics2D object, the actual > implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the > public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the > bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the > superclass and interfaces to find a public class or interface that defines the target > setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it > were non public. > > Can you extend this check to test if the class is in a package exported by its module. For the > example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export > sun.java2d to everyone. Here is a code fragment to test this: > > Class clazz = graphicsObj.getClass(); > boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); > > (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). > > If you can incorporate this check into the bridge then I suspect you'll find most of the examples > will work. Yes, I understand (not being able to use methods in an unexported type's instance, hence the need to find an accessible member in a superclass, which means to have a need to also access protected members in the superclass) and that is actually my current approach. However, I started out with reflecting Fields first and see, whether I can reflectively get access. The rewritten method resolution would follow next, which would allow me to tackle that warning and see whether I can get rid of it. However, before going a wrong route I would like to learn what the "official" Java 9 solution would be and try to implement that. ---rony From mandy.chung at oracle.com Thu Jan 18 19:35:11 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 18 Jan 2018 11:35:11 -0800 Subject: [10] RFR JDK-8194229: tools/jmod/JmodTest.jtr fails when no privilege to create sym link on windows Message-ID: This is a simple test-only fix. IOException may be thrown when a symlink is failed to create on windows. The test should handle such case in addition to UOE. Webrev at: http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8194229/webrev.00/ Mandy From Alan.Bateman at oracle.com Thu Jan 18 19:55:05 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Jan 2018 19:55:05 +0000 Subject: [10] RFR JDK-8194229: tools/jmod/JmodTest.jtr fails when no privilege to create sym link on windows In-Reply-To: References: Message-ID: <86f21591-b714-4e2d-437c-748012d977be@oracle.com> On 18/01/2018 19:35, mandy chung wrote: > This is a simple test-only fix. IOException may be thrown when > a symlink is failed to create on windows. The test should handle > such case in addition to UOE. > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8194229/webrev.00/ Looks okay to me. From Alan.Bateman at oracle.com Fri Jan 19 13:05:44 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jan 2018 13:05:44 +0000 Subject: Unsafe and JPMS In-Reply-To: References: Message-ID: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> Moving this thread to jigsaw-dev as that is where this topic has been discussed ad nauseam. On 19/01/2018 01:28, Jeremy Manson wrote: > Hey folks, > > I know this has come up before, but I wanted to revive the question. As we > play more and more with JPMS and Java 9, we're finding more and more places > where Unsafe is now the only feasible way to achieve what we want to > achieve. > > The big use case is, of course, access to the internals of modularized > libraries in ways that would have used standard reflection in Java 8 and > earlier, but for which we now fail with an exception. Are they really failing with exceptions? I ask because the standard and JDK modules open most of their packages in JDK 9 and JDK 10 for so-called "deep reflection". If there are libraries on the class path hacking into JDK internals then they have a good chance that they will continue to work, assuming the internals haven't changed. There may be a warning of course but that is to help identify code that may break in the future. > > The answer of adding command line flags doesn't really work for general > purpose libraries. If I want to do this in (for example) Guava, asking > every Guava user to add add-opens on the command line is a complete > non-starter. Many people have no idea what libraries they are using. > > The answer of creating an agent is also a non-starter for similar reasons. > > The answer of using privateLookupIn assumes that you have access to or > control over the target class, which is often not the case, especially if > you are introspecting into JDK internals. > > As people within Google try Java 9, this is coming up in quite a bit of our > infrastructure, especially in diagnostic code. The latest place I've found > it was a tool that printed out / counted the number of objects reachable > from a root object; it died when it got into a JDK library. JVM TI is the API for walking the heap of a running VM. That should work as before. Diagnostic tools doing heap walking in Java feels like something for a java agent rather than a general purpose library on the class path (ignoring serialization for now as that's a discussion in itself). If the tooling is using reflection to potentially access every field of every object on the heap then it may need changes (maybe now if it finds itself walking references to objects of classes in packages that are new in JDK 9). The Instrumentation API has the power to do that of course as it can open any package to anyone. This means it can get full-power Lookup to any class in the JDK with privateLookupIn. Agents need to guard this capability closely of course. As regards starting java agents then one addition in JDK 9 to look at is the Launcher-Agent-Class attribute that executable JAR files including an agent can use. > : > > So... OpenJDK has this commitment not to replace Unsafe without providing > supported replacements for its functionality and a transition period. Does > that include the functionality that Unsafe can break module encapsulation? I'm not aware of any current proposals to remove Unsafe. I'm sure degrading Unsafe will come up once Panama and maybe Valhalla are further along. For now, I think the right thing is to continue the effort to identify candidate APIs (where it makes sense) so that code hacking into the JDK today can migrate in the future. This was the motivation for many new APIs in JDK 9 and I've no doubt there will be more before the JDK modules are fully encapsulated. -Alan. From Alan.Bateman at oracle.com Fri Jan 19 14:38:16 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jan 2018 14:38:16 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> Message-ID: <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> On 18/01/2018 15:07, Rony G. Flatscher wrote: > An attachment in the email has been found to contain executable code > and has been removed. > > File removed : java9modules.zip, zip,cmd > ------------------------------------------------------------------------ > > Dear Alan: > > tried to come up with a "cleaner" version to zip it up, however the > error would not occur there. The attachment was dropped too. When you say "the error would not occur there" then do you mean it won't compile? I wouldn't expect the test to compile with a reference to the protected member so maybe the issue was that the code was compiled in a different way and so only found when you ran. -Alan From jeremymanson at google.com Fri Jan 19 18:31:30 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 19 Jan 2018 10:31:30 -0800 Subject: Unsafe and JPMS In-Reply-To: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> References: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> Message-ID: On Fri, Jan 19, 2018 at 5:05 AM, Alan Bateman wrote: > Moving this thread to jigsaw-dev as that is where this topic has been > discussed ad nauseam. Sorry to beat this to death. The engineering time we're spending on the Java 9 transition is very costly. I have a responsibility to make sure I'm doing what I can to mitigate future costs without concomitant benefits. On 19/01/2018 01:28, Jeremy Manson wrote: > >> Hey folks, >> >> I know this has come up before, but I wanted to revive the question. As >> we >> play more and more with JPMS and Java 9, we're finding more and more >> places >> where Unsafe is now the only feasible way to achieve what we want to >> achieve. >> >> The big use case is, of course, access to the internals of modularized >> libraries in ways that would have used standard reflection in Java 8 and >> earlier, but for which we now fail with an exception. >> > Are they really failing with exceptions? I ask because the standard and > JDK modules open most of their packages in JDK 9 and JDK 10 for so-called > "deep reflection". If there are libraries on the class path hacking into > JDK internals then they have a good chance that they will continue to work, > assuming the internals haven't changed. There may be a warning of course > but that is to help identify code that may break in the future. 1) testStaticsDiscovery(com.google.common.profile.HeapInspectorTest)java.lang.reflect.InaccessibleObjectException: Unable to make field final jdk.internal.loader.URLClassPath jdk.internal.loader.ClassLoaders$AppClassLoader.ucp accessible: module java.base does not "opens jdk.internal.loader" to unnamed module @3e11f9e9; did you mean --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:345) at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281) at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176) at java.base/java.lang.reflect.Field.setAccessible(Field.java:170) at com.google.common.profile.HeapInspector.getClassFields(HeapInspector.java:1234) > The answer of adding command line flags doesn't really work for general >> purpose libraries. If I want to do this in (for example) Guava, asking >> every Guava user to add add-opens on the command line is a complete >> non-starter. Many people have no idea what libraries they are using. >> >> The answer of creating an agent is also a non-starter for similar reasons. >> >> The answer of using privateLookupIn assumes that you have access to or >> control over the target class, which is often not the case, especially if >> you are introspecting into JDK internals. >> >> As people within Google try Java 9, this is coming up in quite a bit of >> our >> infrastructure, especially in diagnostic code. The latest place I've >> found >> it was a tool that printed out / counted the number of objects reachable >> from a root object; it died when it got into a JDK library. >> > JVM TI is the API for walking the heap of a running VM. That should work > as before. > > Diagnostic tools doing heap walking in Java feels like something for a > java agent rather than a general purpose library on the class path > (ignoring serialization for now as that's a discussion in itself). If the > tooling is using reflection to potentially access every field of every > object on the heap then it may need changes (maybe now if it finds itself > walking references to objects of classes in packages that are new in JDK > 9). The Instrumentation API has the power to do that of course as it can > open any package to anyone. This means it can get full-power Lookup to any > class in the JDK with privateLookupIn. Agents need to guard this capability > closely of course. As regards starting java agents then one addition in JDK 9 to look at is > the Launcher-Agent-Class attribute that executable JAR files including an > agent can use. > If Launcher-Agent-Class is for executable JAR files, then that's basically a non-starter for library classes, too. Also, it's worth pointing out that even if it feels wrong to you, it has worked in the platform for many years - the cost I'm talking about was written for Java 4, and hasn't needed to be changed since then. I also think that diagnostic code is a red herring here. The OSS use cases I pointed to are not in diagnostic code. So... OpenJDK has this commitment not to replace Unsafe without providing >> supported replacements for its functionality and a transition period. >> Does >> that include the functionality that Unsafe can break module encapsulation? >> > I'm not aware of any current proposals to remove Unsafe. I'm sure > degrading Unsafe will come up once Panama and maybe Valhalla are further > along. > This was a goal until JEP 260 came along. My read on this was that it was simply a matter of time before it came up again. For now, I think the right thing is to continue the effort to identify > candidate APIs (where it makes sense) so that code hacking into the JDK > today can migrate in the future. This was the motivation for many new APIs > in JDK 9 and I've no doubt there will be more before the JDK modules are > fully encapsulated. I generally agree with the principle of making real APIs to do what we need to do with Unsafe and migrating. My concern is not the removal of Unsafe, it's that the replacement won't be equivalent if it doesn't let you cross module boundaries. Jeremy From blackdrag at gmx.org Fri Jan 19 19:53:32 2018 From: blackdrag at gmx.org (Jochen Theodorou) Date: Fri, 19 Jan 2018 20:53:32 +0100 Subject: Unsafe and JPMS In-Reply-To: References: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> Message-ID: <0383f3e2-c999-0813-6c18-fd4bfcd0e0e3@gmx.org> On 19.01.2018 19:31, Jeremy Manson wrote: [...] > I generally agree with the principle of making real APIs to do what we need > to do with Unsafe and migrating. My concern is not the removal of Unsafe, > it's that the replacement won't be equivalent if it doesn't let you cross > module boundaries. It won't be equivalent. It can't be or the module system enforced encapsulation, the big deal of the jpms, would be rendered useless. But maybe the alternatives are good enough for you. You will still have to identify the problems points and see if you can replacement with something else. bye Jochen From Alan.Bateman at oracle.com Fri Jan 19 19:52:35 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jan 2018 19:52:35 +0000 Subject: Unsafe and JPMS In-Reply-To: References: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> Message-ID: <228fd1b2-45ea-0d2c-4e45-b9e1af2be175@oracle.com> On 19/01/2018 18:31, Jeremy Manson wrote: > : > > 1) > testStaticsDiscovery(com.google.common.profile.HeapInspectorTest)java.lang.reflect.InaccessibleObjectException: > Unable to make field final jdk.internal.loader.URLClassPath > jdk.internal.loader.ClassLoaders$AppClassLoader.ucp accessible: module > java.base does not "opens jdk.internal.loader" to unnamed module > @3e11f9e9; did you mean > --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED > at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:345) > at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281) > at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176) > at java.base/java.lang.reflect.Field.setAccessible(Field.java:170) > at > com.google.common.profile.HeapInspector.getClassFields(HeapInspector.java:1234) The class name hints that this is a HeapInspector walking the object graph. I assume it's not deliberately trying to hack the ucp field (btw: is the "did you mean ..." line a Google customization?) The tool may have worked unchanged since JDK 1.4 but it now finds itself in a new world where there are modules that are trying to encapsulate their internals. In this case the java.base module does not want to open jdk.internal.loader (it's new in JDK 9 so this is why it is not open). This is the tension that is discussed here many times. All I'm saying is that the path for diagnostic tooling like this is the tool agent route. Tool agents get encapsulation busting powers at the cost of using tool APIs and being deployed as agents. > > If Launcher-Agent-Class is for executable JAR files, then that's > basically a non-starter for library classes, too. > > Also, it's worth pointing out that even if it feels wrong to you, it > has worked in the platform for many years - the cost I'm talking about > was written for Java 4, and hasn't needed to be changed since then. > > I also think that diagnostic code is a red herring here.? The OSS use > cases I pointed to are not in diagnostic code. Sure, diagnostics tools are special but at least there are APIs and deployment options for these types of tools. One of the libraries you listed seems to a serialization library. The conflict between legacy serialization and encapsulation is irreconcilable. While somewhat sad, this means that custom serialization libraries need to use the updated ReflectionFactory APIs in conjunction with unsafe. The API additions are in JDK 9 and were back-ported to JDK 8u? to make it easier for serialization libraries that want to compile to JDK 8. I can't tell if the library you listed has been updated to use these APIs or not. -Alan From jeremymanson at google.com Sat Jan 20 00:07:50 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 19 Jan 2018 16:07:50 -0800 Subject: Unsafe and JPMS In-Reply-To: <228fd1b2-45ea-0d2c-4e45-b9e1af2be175@oracle.com> References: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> <228fd1b2-45ea-0d2c-4e45-b9e1af2be175@oracle.com> Message-ID: On Fri, Jan 19, 2018 at 11:52 AM, Alan Bateman wrote: > On 19/01/2018 18:31, Jeremy Manson wrote: > > : > > 1) testStaticsDiscovery(com.google.common.profile.HeapInspector > Test)java.lang.reflect.InaccessibleObjectException: Unable to make field > final jdk.internal.loader.URLClassPath jdk.internal.loader.ClassLoaders$AppClassLoader.ucp > accessible: module java.base does not "opens jdk.internal.loader" to > unnamed module @3e11f9e9; did you mean --add-opens=java.base/jdk.inte > rnal.loader=ALL-UNNAMED > at java.base/java.lang.reflect.AccessibleObject.checkCanSetAcce > ssible(AccessibleObject.java:345) > at java.base/java.lang.reflect.AccessibleObject.checkCanSetAcce > ssible(AccessibleObject.java:281) > at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176) > at java.base/java.lang.reflect.Field.setAccessible(Field.java:170) > at com.google.common.profile.HeapInspector.getClassFields(HeapI > nspector.java:1234) > > The class name hints that this is a HeapInspector walking the object > graph. I assume it's not deliberately trying to hack the ucp field (btw: is > the "did you mean ..." line a Google customization?) > Correct - it is just walking the transitively reachable fields. And yes, we stuck that in there to clarify what you could do to work around. (We'd be happy to upstream the patch if folks think it would be desirable - it's not very big). The tool may have worked unchanged since JDK 1.4 but it now finds itself in > a new world where there are modules that are trying to encapsulate their > internals. In this case the java.base module does not want to open > jdk.internal.loader (it's new in JDK 9 so this is why it is not open). This > is the tension that is discussed here many times. All I'm saying is that > the path for diagnostic tooling like this is the tool agent route. Tool > agents get encapsulation busting powers at the cost of using tool APIs and > being deployed as agents. > Right. I guess the question I would have about that is: if I am using an API that is explicitly called "Unsafe", then aren't I basically saying that I'm willing to live without the benefits of encapsulation? In this case, the tradeoff is that we have to make tools that are significantly harder to write and deploy. If I actually have to go down to JVMTI, it's even worse - I would have to write platform specific code in a different language in order to accomplish what I want. And using Unsafe for diagnostic purposes violates none of the stated goals of JPMS, as far as I can tell. (I'm complaining, but we've made deploying agents very easy at Google. I'm mostly dreading the idea that we would have to rewrite it in a way that would take significant time or be harder for people to understand, or that external developers would have to worry about the deployment issues, given the hoops we had to jump through to make deploying agents easy.) If Launcher-Agent-Class is for executable JAR files, then that's basically > a non-starter for library classes, too. > > Also, it's worth pointing out that even if it feels wrong to you, it has > worked in the platform for many years - the cost I'm talking about was > written for Java 4, and hasn't needed to be changed since then. > > I also think that diagnostic code is a red herring here. The OSS use > cases I pointed to are not in diagnostic code. > > Sure, diagnostics tools are special but at least there are APIs and > deployment options for these types of tools. > > One of the libraries you listed seems to a serialization library. The > conflict between legacy serialization and encapsulation is irreconcilable. > While somewhat sad, this means that custom serialization libraries need to > use the updated ReflectionFactory APIs in conjunction with unsafe. The API > additions are in JDK 9 and were back-ported to JDK 8u to make it easier > for serialization libraries that want to compile to JDK 8. I can't tell if > the library you listed has been updated to use these APIs or not. > The critical thing there is that these things are going to receive continued support for module-busting behavior. Which is really what I want to make sure happens - all of those APIs are in jdk.unsupported. :) The other place I saw that this came up is the cglib code generation library, which now uses Unsafe to access ClassLoader.defineClass (for any ClassLoader). This is useful for generating classes for dependency injection / mocking / that kind of thing. It is a violation of encapsulation, but it is done with a very specific software engineering goal in mind. It was never a terribly good approach, but it a library that is now depended upon by a very large amount of code in the wild. (I've thought it might be interesting to replace dynamic frameworks like these with something based on invokedynamic, but I'm not sure how we would indicate which method calls we want to replace with indys without changes to the platform we probably don't want to make.) Jeremy From Alan.Bateman at oracle.com Sat Jan 20 08:30:41 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 20 Jan 2018 08:30:41 +0000 Subject: Unsafe and JPMS In-Reply-To: References: <8adccf49-743e-00da-8bf0-26e4a7ce511c@oracle.com> <228fd1b2-45ea-0d2c-4e45-b9e1af2be175@oracle.com> Message-ID: <1a85097c-5a93-5132-a22c-d4e64e566155@oracle.com> On 20/01/2018 00:07, Jeremy Manson wrote: > : > > The other place I saw that this came up is the cglib code generation > library, which now uses Unsafe to access ClassLoader.defineClass (for > any ClassLoader).? This is useful for generating classes for > dependency injection / mocking / that kind of thing.? It is a > violation of encapsulation, but it is done with a very specific > software engineering goal in mind.? It was never a terribly good > approach, but it a library that is now depended upon by a very large > amount of code in the wild. > DI and other cases with frameworks doing code injection should be looking at Lookup.defineClass. Frameworks can use this API to inject a class into an existing runtime package once they have a Lookup object with the appropriate access. it's not all use-cases but it's the only supported API in the platform for doing this kind of thing. -Alan From peter.levart at gmail.com Mon Jan 22 09:58:16 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jan 2018 10:58:16 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> Message-ID: <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Hi Rony, On 01/18/2018 04:11 PM, Rony G. Flatscher wrote: > On 18.01.2018 10:58, Alan Bateman wrote: >> On 17/01/2018 18:53, Rony G. Flatscher wrote: >>> : >>> >>> Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a >>> module, but having a need to access public types from exported packages and get reflective access >>> to objects supertype's protected members? >> I think it would be better to start with public members as protected is complicated (and hasn't >> changed with modules once you establish the class declaring the member is accessible). >> >> For your example, you've got a reference to a java.awt.Graphics2D object, the actual >> implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the >> public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the >> bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the >> superclass and interfaces to find a public class or interface that defines the target >> setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it >> were non public. >> >> Can you extend this check to test if the class is in a package exported by its module. For the >> example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export >> sun.java2d to everyone. Here is a code fragment to test this: >> >> Class clazz = graphicsObj.getClass(); >> boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); >> >> (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). >> >> If you can incorporate this check into the bridge then I suspect you'll find most of the examples >> will work. > Yes, I understand (not being able to use methods in an unexported type's instance, hence the need to > find an accessible member in a superclass, which means to have a need to also access protected > members in the superclass) and that is actually my current approach. However, I started out with > reflecting Fields first and see, whether I can reflectively get access. > > The rewritten method resolution would follow next, which would allow me to tackle that warning and > see whether I can get rid of it. However, before going a wrong route I would like to learn what the > "official" Java 9 solution would be and try to implement that. > > ---rony Yes, I think you are dealing with two problems here which you have been using the same solution for in the past. The 1st thing you have been doing incorrectly for Java 9, as Alan explained, is the idiom: o.getClass().getMethod(...) and the 2nd is that you are trying to access protected members on behalf of some other class which is a subclass of the protected member's declaring class. The 1st problem has different solutions which are all doable in Java 9, since you are dealing within the confines of public types, public members and exported packages. One solution is to search for the most specific member in the inheritance hierarchy which is also accessible (declared in public type in exported package) which is what Alan suggests. There might also be another elegant solution which requires some re-design of your Rexx interpreter.? When you deal with reference values in Rexx (the values that refer to objects in Java), you could track not only the value itself but also the "static" type of that value. A reference value is always obtained either by calling a constructor, accessing a field (either static or instance), by calling a method (static or instance) or by accessing an element of some array: - calling constructor: the "static type" is the class upon which the constructor has been called - accessing a field: the "static type" is the type of the field (i.e. Field.getDeclaringClass()) - calling a method: the "static type" is the return type of the method (i.e. Method.getReturnType()) - accessing an element of some array: the "static type" is the array's "static type"'s component type (i.e. Class.getComponentType() invoked on array's "static type" Class). When you take the "static" type as the starting Class when searching for a public member with standard Class.getMethod() or Class.getField(), you would then get the correct publicly accessible reflected member. With a caveat that this only works when there's no generics involved. If there's generics, the logic to compute the correct static type is more involved and would sometimes require passing the generic type parameters (when invoking constructors of generic classes or generic methods) in the syntax of your Rexx language. So you may or may not want to do that. Perhaps some library for deep resolving could be of help here (Google Guava has some support for that). I guess searching for the most specific member in the hierarchy that is also accessible is your best bet currently if the goal is to be syntactically backwards compatible in the Rexx language. The 2nd problem is not trivial as you want to access a protected member on behalf of some other sub-class of the member's declaring class which is not cooperating (voluntarily handing you an instance of its Lookup object). This currently requires the package containing the member's declaring class to be opened at least to you (the Rexx interpreter) and using the member.setAccessible(true) trick or MethodHandles.privateLookupIn(declaringClass) equivalent for method handles. Which is awkward because libraries packed as modules would normally not specify that in their module descriptors and system modules don't either. So you are left with either --add-opens command line switches or deploying a javaagent to the JVM and using it's API point java.lang.instrument.Instrumentation#redefineModule to add opens to modules that way. Both approaches are not elegant, but that's what is currently available, I think. Regards, Peter > > From Alan.Bateman at oracle.com Mon Jan 22 11:18:34 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jan 2018 11:18:34 +0000 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: On 22/01/2018 09:58, Peter Levart wrote: > : > > The 2nd problem is not trivial as you want to access a protected > member on behalf of some other sub-class of the member's declaring > class which is not cooperating (voluntarily handing you an instance of > its Lookup object). This currently requires the package containing the > member's declaring class to be opened at least to you (the Rexx > interpreter) and using the member.setAccessible(true) trick or > MethodHandles.privateLookupIn(declaringClass) equivalent for method > handles. Which is awkward because libraries packed as modules would > normally not specify that in their module descriptors and system > modules don't either. So you are left with either --add-opens command > line switches or deploying a javaagent to the JVM and using it's API > point java.lang.instrument.Instrumentation#redefineModule to add opens > to modules that way. Both approaches are not elegant, but that's what > is currently available, I think. > I suspect it may be just a misunderstanding. One of Rony's mails had this example: o=.bsf~new("mtest3.Class03A")???????????? -- create Java object, get and assign proxy ooRexx object say "o:" o "o~myClassName:" o~myClassName -- get (static) field value in "mtest1.Class01A", accessible via inheritance I read this as the Rexx script doing the equivalent of "new mtest3.Class03A()", in which case should be no expectation that protected members are accessible to the Rexx code. -Alan. From peter.levart at gmail.com Mon Jan 22 13:04:31 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jan 2018 14:04:31 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: Hi Rony, On 01/22/2018 10:58 AM, Peter Levart wrote: > The 2nd problem is not trivial as you want to access a protected > member on behalf of some other sub-class of the member's declaring > class which is not cooperating (voluntarily handing you an instance of > its Lookup object). This currently requires the package containing the > member's declaring class to be opened at least to you (the Rexx > interpreter) and using the member.setAccessible(true) trick or > MethodHandles.privateLookupIn(declaringClass) equivalent for method > handles. Which is awkward because libraries packed as modules would > normally not specify that in their module descriptors and system > modules don't either. So you are left with either --add-opens command > line switches or deploying a javaagent to the JVM and using it's API > point java.lang.instrument.Instrumentation#redefineModule to add opens > to modules that way. Both approaches are not elegant, but that's what > is currently available, I think. Just one more thing... While solutions for tackling the 2nd problem might seem attractive to use for solving the 1st problem too, I would recommend not doing that. Opening all the packages of public API(s) might inhibit possible optimizations John Rose has been talking about. For reflective access to public API(s) you don't need to open the packages because public API(s) are in exported packages and all the "static" types that are needed to access them (field types, method return an parameter types) are also guaranteed to be part of public API(s) (at least good modules guarantee that). Public API(s) are transitively public. For public API(s) it is just a matter of finding the accessible member in the hierarchy where there will always be at least one. For the 2nd problem, the main difficulty seems to be how to open just the packages that are involved in accessing the protected members on behalf of subclasses hoping that those packages are in minority. Here's one trick by using javaagent. Suppose your Rexx runtime had the following nonpublic class in its heart: package runtime; import java.util.function.BiConsumer; class Opener { ??? private static class Holder { ??????? static BiConsumer, Module> opener; ??? } ??? static void openPackageOfTo(Class clazz, Module module) { ??????? Holder.opener.accept(clazz, module); ??? } } Now if you start the JVM by supplying the -javaagent:agent.jar command line in addition to everything else and pack the following compiled code into agent.jar with the following MANIFEST: Manifest-Version: 1.0 Premain-Class: agent.Agent --- package agent; import java.lang.instrument.ClassFileTransformer; import java.lang.instrument.IllegalClassFormatException; import java.lang.instrument.Instrumentation; import java.lang.reflect.Field; import java.security.ProtectionDomain; import java.util.Map; import java.util.Set; import java.util.function.BiConsumer; public class Agent { ??? private static final String OPENER_BINARY_CLASS_NAME = "runtime/Opener"; ??? private static final String HOLDER_CLASS_NAME = "runtime.Opener$Holder"; ??? private static final String OPENER_FIELD_NAME = "opener"; ??? private static Instrumentation instrumentation; ??? public static void premain(String agentArgs, Instrumentation inst) { ??????? instrumentation = inst; ??????? inst.addTransformer(new ClassFileTransformer() { ??????????? @Override ??????????? public byte[] transform(Module module, ??????????????????????????????????? ClassLoader loader, ??????????????????????????????????? String className, ??????????????????????????????????? Class classBeingRedefined, ??????????????????????????????????? ProtectionDomain protectionDomain, ??????????????????????????????????? byte[] classfileBuffer) throws IllegalClassFormatException { ??????????????? // when runtime.Opener starts loading... ??????????????? if (className.equals(OPENER_BINARY_CLASS_NAME) && classBeingRedefined == null) { ??????????????????? try { ??????????????????????? // ...load runtime.Opener$Holder upfront using the same classloader ??????????????????????? Class holderClass = Class.forName(HOLDER_CLASS_NAME, true, loader); ??????????????????????? // find the runtime.Opener$Holder#opener field ??????????????????????? Field openerField = holderClass.getDeclaredField(OPENER_FIELD_NAME); ??????????????????????? // and make it accessible ??????????????????????? openerField.setAccessible(true); ??????????????????????? // inject the BiConsumer ??????????????????????? openerField.set(null, (BiConsumer, Module>) Agent::openPackageOfTo); ??????????????????? } catch (ReflectiveOperationException e) { ??????????????????????? throw new InternalError(e); ??????????????????? } ??????????????? } ??????????????? // perform no actual transformation ??????????????? return null; ??????????? } ??????? }, false); ??? } ??? static void openPackageOfTo(Class clazz, Module module) { ??????? String pn = clazz.getPackageName(); ??????? System.out.println("Opening package " + pn + " to " + module); ??????? instrumentation.redefineModule( ??????????? clazz.getModule(), ??????????? Set.of(), ??????????? Map.of(), ??????????? Map.of(pn, Set.of(module)), // extra opens ??????????? Set.of(), ??????????? Map.of() ??????? ); ??? } } With such Rexx runtime specific helper agent jar you can extend the controlled power of java agent to your Rexx interpreter so you now have the power to dynamically add just those opens that are absolutely necessary for performing the accesses to protected members. Regards, Peter From peter.levart at gmail.com Mon Jan 22 13:09:17 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jan 2018 14:09:17 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: <767f9663-b3fd-fc4e-2490-ca155955f079@gmail.com> On 01/22/2018 12:18 PM, Alan Bateman wrote: > On 22/01/2018 09:58, Peter Levart wrote: >> : >> >> The 2nd problem is not trivial as you want to access a protected >> member on behalf of some other sub-class of the member's declaring >> class which is not cooperating (voluntarily handing you an instance >> of its Lookup object). This currently requires the package containing >> the member's declaring class to be opened at least to you (the Rexx >> interpreter) and using the member.setAccessible(true) trick or >> MethodHandles.privateLookupIn(declaringClass) equivalent for method >> handles. Which is awkward because libraries packed as modules would >> normally not specify that in their module descriptors and system >> modules don't either. So you are left with either --add-opens command >> line switches or deploying a javaagent to the JVM and using it's API >> point java.lang.instrument.Instrumentation#redefineModule to add >> opens to modules that way. Both approaches are not elegant, but >> that's what is currently available, I think. >> > I suspect it may be just a misunderstanding. One of Rony's mails had > this example: > > o=.bsf~new("mtest3.Class03A")???????????? -- create Java object, get > and assign proxy ooRexx object > say "o:" o "o~myClassName:" o~myClassName -- get (static) field value > in "mtest1.Class01A", accessible via inheritance > > I read this as the Rexx script doing the equivalent of "new > mtest3.Class03A()", in which case should be no expectation that > protected members are accessible to the Rexx code. > > -Alan. I was asking myself the same question, yes. If Rony wants to call protected methods on "behalf" of subclasses, then Rexx runtime has to have features to subclass existing Java classes. @Rony: So why does Rexx script want to call protected members? Is that because Rexx can extend existing Java classes? Regards, Peter From peter.levart at gmail.com Mon Jan 22 13:33:03 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jan 2018 14:33:03 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: @Potential implementer of below trick BEWARE! While I tried to be smart by "injecting" special java agent powers into designated trusted class, the presented mechanism is NOT SAFE as I identify the class only by it's name. An attacker might create it's own pair of classes with same names (runtime.Opener, runtime.Opener$Holder) loaded in some isolated class loader and the presented agent will happily inject the powers into the attacker's runtime.Opener$Holder. For safe variant of such agent, the Opener$Holder class would have to prove the authenticity to the Agent 1st. The mechanism of such authentication will be left as an exercise to the reader... Regards, Peter On 01/22/2018 02:04 PM, Peter Levart wrote: > Hi Rony, > > On 01/22/2018 10:58 AM, Peter Levart wrote: >> The 2nd problem is not trivial as you want to access a protected >> member on behalf of some other sub-class of the member's declaring >> class which is not cooperating (voluntarily handing you an instance >> of its Lookup object). This currently requires the package containing >> the member's declaring class to be opened at least to you (the Rexx >> interpreter) and using the member.setAccessible(true) trick or >> MethodHandles.privateLookupIn(declaringClass) equivalent for method >> handles. Which is awkward because libraries packed as modules would >> normally not specify that in their module descriptors and system >> modules don't either. So you are left with either --add-opens command >> line switches or deploying a javaagent to the JVM and using it's API >> point java.lang.instrument.Instrumentation#redefineModule to add >> opens to modules that way. Both approaches are not elegant, but >> that's what is currently available, I think. > > Just one more thing... While solutions for tackling the 2nd problem > might seem attractive to use for solving the 1st problem too, I would > recommend not doing that. Opening all the packages of public API(s) > might inhibit possible optimizations John Rose has been talking about. > For reflective access to public API(s) you don't need to open the > packages because public API(s) are in exported packages and all the > "static" types that are needed to access them (field types, method > return an parameter types) are also guaranteed to be part of public > API(s) (at least good modules guarantee that). Public API(s) are > transitively public. For public API(s) it is just a matter of finding > the accessible member in the hierarchy where there will always be at > least one. > > For the 2nd problem, the main difficulty seems to be how to open just > the packages that are involved in accessing the protected members on > behalf of subclasses hoping that those packages are in minority. > Here's one trick by using javaagent. Suppose your Rexx runtime had the > following nonpublic class in its heart: > > package runtime; > > import java.util.function.BiConsumer; > > class Opener { > ??? private static class Holder { > ??????? static BiConsumer, Module> opener; > ??? } > > ??? static void openPackageOfTo(Class clazz, Module module) { > ??????? Holder.opener.accept(clazz, module); > ??? } > } > > > Now if you start the JVM by supplying the -javaagent:agent.jar command > line in addition to everything else and pack the following compiled > code into agent.jar with the following MANIFEST: > > Manifest-Version: 1.0 > Premain-Class: agent.Agent > > --- > package agent; > > import java.lang.instrument.ClassFileTransformer; > import java.lang.instrument.IllegalClassFormatException; > import java.lang.instrument.Instrumentation; > import java.lang.reflect.Field; > import java.security.ProtectionDomain; > import java.util.Map; > import java.util.Set; > import java.util.function.BiConsumer; > > public class Agent { > ??? private static final String OPENER_BINARY_CLASS_NAME = > "runtime/Opener"; > ??? private static final String HOLDER_CLASS_NAME = > "runtime.Opener$Holder"; > ??? private static final String OPENER_FIELD_NAME = "opener"; > > ??? private static Instrumentation instrumentation; > > ??? public static void premain(String agentArgs, Instrumentation inst) { > ??????? instrumentation = inst; > ??????? inst.addTransformer(new ClassFileTransformer() { > ??????????? @Override > ??????????? public byte[] transform(Module module, > ??????????????????????????????????? ClassLoader loader, > ??????????????????????????????????? String className, > ??????????????????????????????????? Class classBeingRedefined, > ??????????????????????????????????? ProtectionDomain protectionDomain, > ??????????????????????????????????? byte[] classfileBuffer) throws > IllegalClassFormatException { > > ??????????????? // when runtime.Opener starts loading... > ??????????????? if (className.equals(OPENER_BINARY_CLASS_NAME) && > classBeingRedefined == null) { > ??????????????????? try { > ??????????????????????? // ...load runtime.Opener$Holder upfront using > the same classloader > ??????????????????????? Class holderClass = > Class.forName(HOLDER_CLASS_NAME, true, loader); > ??????????????????????? // find the runtime.Opener$Holder#opener field > ??????????????????????? Field openerField = > holderClass.getDeclaredField(OPENER_FIELD_NAME); > ??????????????????????? // and make it accessible > ??????????????????????? openerField.setAccessible(true); > ??????????????????????? // inject the BiConsumer > ??????????????????????? openerField.set(null, (BiConsumer, > Module>) Agent::openPackageOfTo); > ??????????????????? } catch (ReflectiveOperationException e) { > ??????????????????????? throw new InternalError(e); > ??????????????????? } > ??????????????? } > > ??????????????? // perform no actual transformation > ??????????????? return null; > ??????????? } > ??????? }, false); > ??? } > > ??? static void openPackageOfTo(Class clazz, Module module) { > ??????? String pn = clazz.getPackageName(); > ??????? System.out.println("Opening package " + pn + " to " + module); > ??????? instrumentation.redefineModule( > ??????????? clazz.getModule(), > ??????????? Set.of(), > ??????????? Map.of(), > ??????????? Map.of(pn, Set.of(module)), // extra opens > ??????????? Set.of(), > ??????????? Map.of() > ??????? ); > ??? } > } > > > With such Rexx runtime specific helper agent jar you can extend the > controlled power of java agent to your Rexx interpreter so you now > have the power to dynamically add just those opens that are absolutely > necessary for performing the accesses to protected members. > > > Regards, Peter > From Rony.Flatscher at wu.ac.at Mon Jan 22 15:18:03 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Mon, 22 Jan 2018 16:18:03 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> Message-ID: <3c4c72cb-58f4-b889-ff47-9e2db941eb89@wu.ac.at> On 19.01.2018 15:38, Alan Bateman wrote: > On 18/01/2018 15:07, Rony G. Flatscher wrote: >> An attachment in the email has been found to contain executable code and has been removed. >> >> File removed : java9modules.zip, zip,cmd >> ---------------------------------------------------------------------------------------------------- >> >> Dear Alan: >> >> tried to come up with a "cleaner" version to zip it up, however the error would not occur there. > The attachment was dropped too. OK, next time, I will use Dropbox or the like instead. > When you say "the error would not occur there" then do you mean it won't compile? I wouldn't > expect the test to compile with a reference to the protected member so maybe the issue was that > the code was compiled in a different way and so only found when you ran. Looked into it and you are probably right, the class file was a few minutes older than the latest version of the java file. Deleting the class file and trying to recompile yields the expected error "...has protected access in Class01A..." and there is no public getter defined for that field in any of the classes in the exported packages. ---rony From Roger.Riggs at Oracle.com Mon Jan 22 15:24:42 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 22 Jan 2018 10:24:42 -0500 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <3c4c72cb-58f4-b889-ff47-9e2db941eb89@wu.ac.at> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> <3c4c72cb-58f4-b889-ff47-9e2db941eb89@wu.ac.at> Message-ID: <50d12613-ed48-78a1-a684-eab599fcb327@Oracle.com> Hi, Contributions need to be submitted using the OpenJDK infrastructure to adhere to the IP requirements. The mail lists shold pass attachments that are text and patches though you may need to be sure your mailer attaches them with the correct mime-types and/or extensions. Roger p.s. The current settings have pass_mime_types: multipart/mixed multipart/alternative text/plain text/x-diff text/x-patch message/rfc822 multipart/signed On 1/22/2018 10:18 AM, Rony G. Flatscher wrote: > On 19.01.2018 15:38, Alan Bateman wrote: >> On 18/01/2018 15:07, Rony G. Flatscher wrote: >>> An attachment in the email has been found to contain executable code and has been removed. >>> >>> File removed : java9modules.zip, zip,cmd >>> ---------------------------------------------------------------------------------------------------- >>> >>> Dear Alan: >>> >>> tried to come up with a "cleaner" version to zip it up, however the error would not occur there. >> The attachment was dropped too. > OK, next time, I will use Dropbox or the like instead. Sorry, only inline, attached or on cr.openjdk.java.net. From Rony.Flatscher at wu.ac.at Mon Jan 22 15:35:34 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Mon, 22 Jan 2018 16:35:34 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: Hi Peter, thank you *very* much also for your kind explanations and even coming up with agent code to demonstrate how one could use that approach! In fact I have been doing a static analysis in the past (since Java 1.1) to determine whether members should be accessible to Rexx, restricting access to public members and to protected inherited members, if accessing them from a dynamically created subclass. One thing I have to make sure is, that I keep compatible with the Java 1.6/6.0 baseline, as there are some deployments where the shops are still employing that environment. This should not really be a problem as I am rewriting the entire reflection part and will be able to employ different approaches for pre-Java 9 deplyoments, where the reflection class employed is dependent on the Java runtime version. Currently I am exploring the implications of the new Java 9 module system, trying to adhere to its rules. In essence the goal is to allow reflectively everything for Rexx peers that a compiled Java program allows for in the Java 9 environment. For that the reflecting class (currently in the unnamed module) goes up the inheritance tree until it finds an exported class and analyses it reflectively. Experimenting with variations of classes residing in different modules with different exports, it is possible to mix-up the relationships, what gets exported to what, and when should protected members in superclasses be accessible and when not (and yes, this part should belong to dynamically created subclasses, which also need adjustments to the module system). Best regards, ---rony On 22.01.2018 10:58, Peter Levart wrote: > Hi Rony, > > On 01/18/2018 04:11 PM, Rony G. Flatscher wrote: >> On 18.01.2018 10:58, Alan Bateman wrote: >>> On 17/01/2018 18:53, Rony G. Flatscher wrote: >>>> : >>>> >>>> Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a >>>> module, but having a need to access public types from exported packages and get reflective access >>>> to objects supertype's protected members? >>> I think it would be better to start with public members as protected is complicated (and hasn't >>> changed with modules once you establish the class declaring the member is accessible). >>> >>> For your example, you've got a reference to a java.awt.Graphics2D object, the actual >>> implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the >>> public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the >>> bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the >>> superclass and interfaces to find a public class or interface that defines the target >>> setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it >>> were non public. >>> >>> Can you extend this check to test if the class is in a package exported by its module. For the >>> example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export >>> sun.java2d to everyone. Here is a code fragment to test this: >>> >>> Class clazz = graphicsObj.getClass(); >>> boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); >>> >>> (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). >>> >>> If you can incorporate this check into the bridge then I suspect you'll find most of the examples >>> will work. >> Yes, I understand (not being able to use methods in an unexported type's instance, hence the need to >> find an accessible member in a superclass, which means to have a need to also access protected >> members in the superclass) and that is actually my current approach. However, I started out with >> reflecting Fields first and see, whether I can reflectively get access. >> >> The rewritten method resolution would follow next, which would allow me to tackle that warning and >> see whether I can get rid of it. However, before going a wrong route I would like to learn what the >> "official" Java 9 solution would be and try to implement that. >> >> ---rony > > Yes, I think you are dealing with two problems here which you have been using the same solution > for in the past. > > The 1st thing you have been doing incorrectly for Java 9, as Alan explained, is the idiom: > o.getClass().getMethod(...) and the 2nd is that you are trying to access protected members on > behalf of some other class which is a subclass of the protected member's declaring class. > > The 1st problem has different solutions which are all doable in Java 9, since you are dealing > within the confines of public types, public members and exported packages. One solution is to > search for the most specific member in the inheritance hierarchy which is also accessible > (declared in public type in exported package) which is what Alan suggests. > > There might also be another elegant solution which requires some re-design of your Rexx > interpreter.? When you deal with reference values in Rexx (the values that refer to objects in > Java), you could track not only the value itself but also the "static" type of that value. A > reference value is always obtained either by calling a constructor, accessing a field (either > static or instance), by calling a method (static or instance) or by accessing an element of some > array: > > - calling constructor: the "static type" is the class upon which the constructor has been called > - accessing a field: the "static type" is the type of the field (i.e. Field.getDeclaringClass()) > - calling a method: the "static type" is the return type of the method (i.e. Method.getReturnType()) > - accessing an element of some array: the "static type" is the array's "static type"'s component > type (i.e. Class.getComponentType() invoked on array's "static type" Class). > > When you take the "static" type as the starting Class when searching for a public member with > standard Class.getMethod() or Class.getField(), you would then get the correct publicly accessible > reflected member. With a caveat that this only works when there's no generics involved. If there's > generics, the logic to compute the correct static type is more involved and would sometimes > require passing the generic type parameters (when invoking constructors of generic classes or > generic methods) in the syntax of your Rexx language. So you may or may not want to do that. > Perhaps some library for deep resolving could be of help here (Google Guava has some support for > that). I guess searching for the most specific member in the hierarchy that is also accessible is > your best bet currently if the goal is to be syntactically backwards compatible in the Rexx language. > > The 2nd problem is not trivial as you want to access a protected member on behalf of some other > sub-class of the member's declaring class which is not cooperating (voluntarily handing you an > instance of its Lookup object). This currently requires the package containing the member's > declaring class to be opened at least to you (the Rexx interpreter) and using the > member.setAccessible(true) trick or MethodHandles.privateLookupIn(declaringClass) equivalent for > method handles. Which is awkward because libraries packed as modules would normally not specify > that in their module descriptors and system modules don't either. So you are left with either > --add-opens command line switches or deploying a javaagent to the JVM and using it's API point > java.lang.instrument.Instrumentation#redefineModule to add opens to modules that way. Both > approaches are not elegant, but that's what is currently available, I think. > > Regards, Peter From Rony.Flatscher at wu.ac.at Mon Jan 22 15:38:00 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Mon, 22 Jan 2018 16:38:00 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: On 22.01.2018 12:18, Alan Bateman wrote: > On 22/01/2018 09:58, Peter Levart wrote: >> : >> >> The 2nd problem is not trivial as you want to access a protected member on behalf of some other >> sub-class of the member's declaring class which is not cooperating (voluntarily handing you an >> instance of its Lookup object). This currently requires the package containing the member's >> declaring class to be opened at least to you (the Rexx interpreter) and using the >> member.setAccessible(true) trick or MethodHandles.privateLookupIn(declaringClass) equivalent for >> method handles. Which is awkward because libraries packed as modules would normally not specify >> that in their module descriptors and system modules don't either. So you are left with either >> --add-opens command line switches or deploying a javaagent to the JVM and using it's API point >> java.lang.instrument.Instrumentation#redefineModule to add opens to modules that way. Both >> approaches are not elegant, but that's what is currently available, I think. >> > I suspect it may be just a misunderstanding. One of Rony's mails had this example: > > o=.bsf~new("mtest3.Class03A")???????????? -- create Java object, get and assign proxy ooRexx object > say "o:" o "o~myClassName:" o~myClassName -- get (static) field value in "mtest1.Class01A", > accessible via inheritance > > I read this as the Rexx script doing the equivalent of "new mtest3.Class03A()", in which case > should be no expectation that protected members are accessible to the Rexx code. Yes, you are right! There need to be a public member that is capable of accessing the protected ones in this case. Obviously I have used too many variants and mixed up the use cases, really sorry for the noise! :( ---rony From Rony.Flatscher at wu.ac.at Mon Jan 22 15:39:01 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Mon, 22 Jan 2018 16:39:01 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <50d12613-ed48-78a1-a684-eab599fcb327@Oracle.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> <3c4c72cb-58f4-b889-ff47-9e2db941eb89@wu.ac.at> <50d12613-ed48-78a1-a684-eab599fcb327@Oracle.com> Message-ID: <106d98f0-5637-89c8-2fb7-9e61d5f46f67@wu.ac.at> Hi, On 22.01.2018 16:24, Roger Riggs wrote: > Contributions need to be submitted using the OpenJDK infrastructure to adhere to the IP requirements. Would you have a link which OpenJFDK infrastructure to use in this case? > The mail lists shold pass attachments that are text and patches though you may need to be > sure your mailer attaches them with the correct mime-types and/or extensions. > > Roger > > p.s. > The current settings have pass_mime_types: > > ?? multipart/mixed > ?? multipart/alternative > ?? text/plain > ?? text/x-diff > ?? text/x-patch > ?? message/rfc822 > ?? multipart/signed Thanks for this information! Regards, ---rony > On 1/22/2018 10:18 AM, Rony G. Flatscher wrote: >> On 19.01.2018 15:38, Alan Bateman wrote: >>> On 18/01/2018 15:07, Rony G. Flatscher wrote: >>>> An attachment in the email has been found to contain executable code and has been removed. >>>> >>>> File removed : java9modules.zip, zip,cmd >>>> ---------------------------------------------------------------------------------------------------- >>>> >>>> >>>> Dear Alan: >>>> >>>> tried to come up with a "cleaner" version to zip it up, however the error would not occur there. >>> The attachment was dropped too. >> OK, next time, I will use Dropbox or the like instead. > Sorry, only inline, attached or on cr.openjdk.java.net. > From Roger.Riggs at Oracle.com Mon Jan 22 16:08:44 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 22 Jan 2018 11:08:44 -0500 Subject: Reflection: how does one access ... In-Reply-To: <106d98f0-5637-89c8-2fb7-9e61d5f46f67@wu.ac.at> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <1018a06b-515b-b96f-7eec-c653a8919e03@wu.ac.at> <4bf02122-c095-e3fd-b947-f12aff927643@oracle.com> <3c4c72cb-58f4-b889-ff47-9e2db941eb89@wu.ac.at> <50d12613-ed48-78a1-a684-eab599fcb327@Oracle.com> <106d98f0-5637-89c8-2fb7-9e61d5f46f67@wu.ac.at> Message-ID: Hi, Attaching the patch as a .patch or .txt file should be sufficient for small contributions and example code. Roger On 1/22/2018 10:39 AM, Rony G. Flatscher wrote: > Hi, > > On 22.01.2018 16:24, Roger Riggs wrote: >> Contributions need to be submitted using the OpenJDK infrastructure to adhere to the IP requirements. > Would you have a link which OpenJFDK infrastructure to use in this case? > >> The mail lists shold pass attachments that are text and patches though you may need to be >> sure your mailer attaches them with the correct mime-types and/or extensions. >> >> Roger >> >> p.s. >> The current settings have pass_mime_types: >> >> ?? multipart/mixed >> ?? multipart/alternative >> ?? text/plain >> ?? text/x-diff >> ?? text/x-patch >> ?? message/rfc822 >> ?? multipart/signed > Thanks for this information! > > Regards, > > ---rony > >> On 1/22/2018 10:18 AM, Rony G. Flatscher wrote: >>> On 19.01.2018 15:38, Alan Bateman wrote: >>>> On 18/01/2018 15:07, Rony G. Flatscher wrote: >>>>> An attachment in the email has been found to contain executable code and has been removed. >>>>> >>>>> File removed : java9modules.zip, zip,cmd >>>>> ---------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> Dear Alan: >>>>> >>>>> tried to come up with a "cleaner" version to zip it up, however the error would not occur there. >>>> The attachment was dropped too. >>> OK, next time, I will use Dropbox or the like instead. >> Sorry, only inline, attached or on cr.openjdk.java.net. >> From peter.levart at gmail.com Tue Jan 23 07:10:53 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 23 Jan 2018 08:10:53 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> Message-ID: <62bf41bf-9621-f254-54da-24f678bf52e9@gmail.com> Hi Rony, On 01/22/18 16:35, Rony G. Flatscher wrote: > Hi Peter, > > thank you *very* much also for your kind explanations and even coming up with agent code to > demonstrate how one could use that approach! > > In fact I have been doing a static analysis in the past (since Java 1.1) to determine whether > members should be accessible to Rexx, restricting access to public members and to protected > inherited members, if accessing them from a dynamically created subclass. So you *are* loading dynamically generated subclasses. It's just that their logic is implemented in Rexx scripting and therefore delegates invocations to Rexx runtime and those invocations include invocations to protected methods of their superclasses. In that case, why don't you come up with some mechanism for dynamically generated classes to hand over their Lookup(s) to Rexx runtime. For example, the Rexx runtime could have the following public registration API: --- package runtime; import java.lang.invoke.MethodHandles; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; public class LookupRegistry { ??? private static final Map, MethodHandles.Lookup> registry = ??????? new ConcurrentHashMap<>(); ??? public static void register(MethodHandles.Lookup lookup) { ??????? if (registry.putIfAbsent(lookup.lookupClass(), lookup) != null) { ??????????? throw new IllegalStateException( ??????????????? "Lookup for " + lookup.lookupClass() + " is already registered"); ??????? } ??? } ??? /* non-public */ static MethodHandles.Lookup getLookup(Class clazz) { ??????? return registry.get(clazz); ??? } } ...then each Rexx dynamically generated class could include the following static initialization block: public class GeneratedClass { ??? static { ??????? runtime.LookupRegistry.register( ??????????? java.lang.invoke.MethodHandles.lookup() ??????? ); ??? } ... That way Rexx runtime could have access to Lookup(s) of each of it's dynamically generated classes as soon as they are loaded and initialized and can use a particular class's Lookup to lookup a MethodHandle for the protected methods of its superclass. It can even simulate a "super.method()" invocation, etc... Regards, Peter > One thing I have to make sure is, that I keep compatible with the Java 1.6/6.0 baseline, as there > are some deployments where the shops are still employing that environment. This should not really be > a problem as I am rewriting the entire reflection part and will be able to employ different > approaches for pre-Java 9 deplyoments, where the reflection class employed is dependent on the Java > runtime version. > > Currently I am exploring the implications of the new Java 9 module system, trying to adhere to its > rules. In essence the goal is to allow reflectively everything for Rexx peers that a compiled Java > program allows for in the Java 9 environment. For that the reflecting class (currently in the > unnamed module) goes up the inheritance tree until it finds an exported class and analyses it > reflectively. > > Experimenting with variations of classes residing in different modules with different exports, it is > possible to mix-up the relationships, what gets exported to what, and when should protected members > in superclasses be accessible and when not (and yes, this part should belong to dynamically created > subclasses, which also need adjustments to the module system). > > Best regards, > > ---rony > > > On 22.01.2018 10:58, Peter Levart wrote: >> Hi Rony, >> >> On 01/18/2018 04:11 PM, Rony G. Flatscher wrote: >>> On 18.01.2018 10:58, Alan Bateman wrote: >>>> On 17/01/2018 18:53, Rony G. Flatscher wrote: >>>>> : >>>>> >>>>> Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a >>>>> module, but having a need to access public types from exported packages and get reflective access >>>>> to objects supertype's protected members? >>>> I think it would be better to start with public members as protected is complicated (and hasn't >>>> changed with modules once you establish the class declaring the member is accessible). >>>> >>>> For your example, you've got a reference to a java.awt.Graphics2D object, the actual >>>> implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the >>>> public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the >>>> bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the >>>> superclass and interfaces to find a public class or interface that defines the target >>>> setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it >>>> were non public. >>>> >>>> Can you extend this check to test if the class is in a package exported by its module. For the >>>> example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export >>>> sun.java2d to everyone. Here is a code fragment to test this: >>>> >>>> Class clazz = graphicsObj.getClass(); >>>> boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); >>>> >>>> (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). >>>> >>>> If you can incorporate this check into the bridge then I suspect you'll find most of the examples >>>> will work. >>> Yes, I understand (not being able to use methods in an unexported type's instance, hence the need to >>> find an accessible member in a superclass, which means to have a need to also access protected >>> members in the superclass) and that is actually my current approach. However, I started out with >>> reflecting Fields first and see, whether I can reflectively get access. >>> >>> The rewritten method resolution would follow next, which would allow me to tackle that warning and >>> see whether I can get rid of it. However, before going a wrong route I would like to learn what the >>> "official" Java 9 solution would be and try to implement that. >>> >>> ---rony >> Yes, I think you are dealing with two problems here which you have been using the same solution >> for in the past. >> >> The 1st thing you have been doing incorrectly for Java 9, as Alan explained, is the idiom: >> o.getClass().getMethod(...) and the 2nd is that you are trying to access protected members on >> behalf of some other class which is a subclass of the protected member's declaring class. >> >> The 1st problem has different solutions which are all doable in Java 9, since you are dealing >> within the confines of public types, public members and exported packages. One solution is to >> search for the most specific member in the inheritance hierarchy which is also accessible >> (declared in public type in exported package) which is what Alan suggests. >> >> There might also be another elegant solution which requires some re-design of your Rexx >> interpreter.? When you deal with reference values in Rexx (the values that refer to objects in >> Java), you could track not only the value itself but also the "static" type of that value. A >> reference value is always obtained either by calling a constructor, accessing a field (either >> static or instance), by calling a method (static or instance) or by accessing an element of some >> array: >> >> - calling constructor: the "static type" is the class upon which the constructor has been called >> - accessing a field: the "static type" is the type of the field (i.e. Field.getDeclaringClass()) >> - calling a method: the "static type" is the return type of the method (i.e. Method.getReturnType()) >> - accessing an element of some array: the "static type" is the array's "static type"'s component >> type (i.e. Class.getComponentType() invoked on array's "static type" Class). >> >> When you take the "static" type as the starting Class when searching for a public member with >> standard Class.getMethod() or Class.getField(), you would then get the correct publicly accessible >> reflected member. With a caveat that this only works when there's no generics involved. If there's >> generics, the logic to compute the correct static type is more involved and would sometimes >> require passing the generic type parameters (when invoking constructors of generic classes or >> generic methods) in the syntax of your Rexx language. So you may or may not want to do that. >> Perhaps some library for deep resolving could be of help here (Google Guava has some support for >> that). I guess searching for the most specific member in the hierarchy that is also accessible is >> your best bet currently if the goal is to be syntactically backwards compatible in the Rexx language. >> >> The 2nd problem is not trivial as you want to access a protected member on behalf of some other >> sub-class of the member's declaring class which is not cooperating (voluntarily handing you an >> instance of its Lookup object). This currently requires the package containing the member's >> declaring class to be opened at least to you (the Rexx interpreter) and using the >> member.setAccessible(true) trick or MethodHandles.privateLookupIn(declaringClass) equivalent for >> method handles. Which is awkward because libraries packed as modules would normally not specify >> that in their module descriptors and system modules don't either. So you are left with either >> --add-opens command line switches or deploying a javaagent to the JVM and using it's API point >> java.lang.instrument.Instrumentation#redefineModule to add opens to modules that way. Both >> approaches are not elegant, but that's what is currently available, I think. >> >> Regards, Peter From Rony.Flatscher at wu.ac.at Tue Jan 23 14:12:19 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 23 Jan 2018 15:12:19 +0100 Subject: Reflection: how does one access a protected member in a superclass reflectively? In-Reply-To: <62bf41bf-9621-f254-54da-24f678bf52e9@gmail.com> References: <8b2e2502-e989-69dc-5a51-47e3102bb390@oracle.com> <8d1afaec-385c-3bc6-4a83-a68cd6a79c83@oracle.com> <61E42714-554F-4E21-99FA-1C5A896AD170@oracle.com> <22e68d92-f85e-6306-1b39-91f86221454d@wu.ac.at> <0cf377cf-c0ae-1af7-e490-93c3cdd4934e@oracle.com> <0ce9faed-9b38-e9b8-18cb-a791aa0ee5f6@oracle.com> <631c0566-e107-6182-51d6-ea4d1ac2f282@gmail.com> <62bf41bf-9621-f254-54da-24f678bf52e9@gmail.com> Message-ID: <6de12a98-e5fe-81e1-0f7b-f5662080f86c@wu.ac.at> Hi Peter, firstly: thank you *very* much again for your kind hints and help! Ad Rexx and Java: the aim is to have Rexx (dynamically typed, caseless) behave 1:1 like a compilable and runnable Java program. Over the course of time I looked into ASM (many, many years ago) to add the ability to create dynamically subclasses. In the meantime Janino is at play as I really needed just the ability to create dynamically simple subclasses (again quite some time ago). This support was needed to allow abstract Java methods (and Java interface classes) to be implemented in ooRexx. It was also implemented such, that concrete Java classes can be subclassed at runtime as well (supplying a list of Java methods that should be interceptable by Rexx methods, but still allow Rexx to invoke the Java methods in superclasses). The reflection infrastructure in the bridge has been developed over the course of more than 15 years. It worked until Java 1.8/8 (introduction of new static and default methods) unchanged for many years. Now with Java 9 clearly the reflection part needs to be redone to work with Java 9 and I take the opportunity to rewrite it altogether in a way (hoping) that the module related problems can be solved via reflection only, such that I can also create stripped down versions of the rewritten reflection part for Java 1.8/8 and another for Java 1.6/6 and 1.7/7. Hence my desire to stick to reflection for the time being. If that is not possible with some of the needed functionality then I would immediately turn to MethodHandles and try to follow your advice. The task of the bridge is in principle is to allow Java programs to be written in Rexx and when the Rexx program executes it should execute as if it was a Java program (if that can be compiled and run)! :) [The bridge allows Java to employ Rexx as a scripting language as well, implementing Apache's BSF and Java's 1.6/6 javax.script.ScriptEngine and the like.] The Java objects to work with reflectively are received by invoking Java methods so are out of control of the bridge. All interactions with Java have been done using the Java reflection mechanism (including with the dynamically created Java classes at runtime) since Java 1.1. Again, thank you very much for your kind help and advice! ---rony P.S.: Currently I have been tracing down accessing public fields in a superclass of a type which package is exported from the unnamed module (Rexx reflection at this stage works out of the unnamed module), but the superclass resides in a non-exported package. A Java program from the unnamed module is able to access those public fields, doing the same with reflection has not been successful yet. Will double-check the example and post it under a new thread (javac 9 creates quite different code for that Java program depending whether using the module or the classpath where the module version uses MethodHandles; however both versions are able to access those public fields, which I cannot access via reflection yet). On 23.01.2018 08:10, Peter Levart wrote: > Hi Rony, > > On 01/22/18 16:35, Rony G. Flatscher wrote: >> Hi Peter, >> >> thank you *very* much also for your kind explanations and even coming up with agent code to >> demonstrate how one could use that approach! >> >> In fact I have been doing a static analysis in the past (since Java 1.1) to determine whether >> members should be accessible to Rexx, restricting access to public members and to protected >> inherited members, if accessing them from a dynamically created subclass. > > So you *are* loading dynamically generated subclasses. It's just that their logic is implemented > in Rexx scripting and therefore delegates invocations to Rexx runtime and those invocations > include invocations to protected methods of their superclasses. > > In that case, why don't you come up with some mechanism for dynamically generated classes to hand > over their Lookup(s) to Rexx runtime. For example, the Rexx runtime could have the following > public registration API: > > --- > package runtime; > > import java.lang.invoke.MethodHandles; > import java.util.Map; > import java.util.concurrent.ConcurrentHashMap; > > public class LookupRegistry { > ??? private static final Map, MethodHandles.Lookup> registry = > ??????? new ConcurrentHashMap<>(); > > ??? public static void register(MethodHandles.Lookup lookup) { > ??????? if (registry.putIfAbsent(lookup.lookupClass(), lookup) != null) { > ??????????? throw new IllegalStateException( > ??????????????? "Lookup for " + lookup.lookupClass() + " is already registered"); > ??????? } > ??? } > > ??? /* non-public */ static MethodHandles.Lookup getLookup(Class clazz) { > ??????? return registry.get(clazz); > ??? } > } > > > ...then each Rexx dynamically generated class could include the following static initialization block: > > > public class GeneratedClass { > ??? static { > ??????? runtime.LookupRegistry.register( > ??????????? java.lang.invoke.MethodHandles.lookup() > ??????? ); > ??? } > ... > > > That way Rexx runtime could have access to Lookup(s) of each of it's dynamically generated classes > as soon as they are loaded and initialized and can use a particular class's Lookup to lookup a > MethodHandle for the protected methods of its superclass. It can even simulate a "super.method()" > invocation, etc... > > > Regards, Peter > >> One thing I have to make sure is, that I keep compatible with the Java 1.6/6.0 baseline, as there >> are some deployments where the shops are still employing that environment. This should not really be >> a problem as I am rewriting the entire reflection part and will be able to employ different >> approaches for pre-Java 9 deplyoments, where the reflection class employed is dependent on the Java >> runtime version. >> >> Currently I am exploring the implications of the new Java 9 module system, trying to adhere to its >> rules. In essence the goal is to allow reflectively everything for Rexx peers that a compiled Java >> program allows for in the Java 9 environment. For that the reflecting class (currently in the >> unnamed module) goes up the inheritance tree until it finds an exported class and analyses it >> reflectively. >> >> Experimenting with variations of classes residing in different modules with different exports, it is >> possible to mix-up the relationships, what gets exported to what, and when should protected members >> in superclasses be accessible and when not (and yes, this part should belong to dynamically created >> subclasses, which also need adjustments to the module system). >> >> Best regards, >> >> ---rony >> >> >> On 22.01.2018 10:58, Peter Levart wrote: >>> Hi Rony, >>> >>> On 01/18/2018 04:11 PM, Rony G. Flatscher wrote: >>>> On 18.01.2018 10:58, Alan Bateman wrote: >>>>> On 17/01/2018 18:53, Rony G. Flatscher wrote: >>>>>> : >>>>>> >>>>>> Would you have concrete suggestions for this use-case, i.e. a framework that is not part of a >>>>>> module, but having a need to access public types from exported packages and get reflective access >>>>>> to objects supertype's protected members? >>>>> I think it would be better to start with public members as protected is complicated (and hasn't >>>>> changed with modules once you establish the class declaring the member is accessible). >>>>> >>>>> For your example, you've got a reference to a java.awt.Graphics2D object, the actual >>>>> implementation type is sun.java2d.SunGraphics2D. The user is attempting to invoke one of the >>>>> public setRenderingHint methods that Graphics2D defines. You said in one of your mails that the >>>>> bridge "iterates over all its superclasses" which I take to mean that it recursively looks at the >>>>> superclass and interfaces to find a public class or interface that defines the target >>>>> setRenderingHint method. In the example, I expect it would skip sun.java2d.SunGraphics2D if it >>>>> were non public. >>>>> >>>>> Can you extend this check to test if the class is in a package exported by its module. For the >>>>> example, sun.java2d.SunGraphics2D is in the java.desktop module and this module does not export >>>>> sun.java2d to everyone. Here is a code fragment to test this: >>>>> >>>>> Class clazz = graphicsObj.getClass(); >>>>> boolean isExportedToAll = clazz.getModule().isExported(clazz.getPackageName()); >>>>> >>>>> (I'm deliberately avoiding the 2-arg isExported to keep things simple for this discussion). >>>>> >>>>> If you can incorporate this check into the bridge then I suspect you'll find most of the examples >>>>> will work. >>>> Yes, I understand (not being able to use methods in an unexported type's instance, hence the need to >>>> find an accessible member in a superclass, which means to have a need to also access protected >>>> members in the superclass) and that is actually my current approach. However, I started out with >>>> reflecting Fields first and see, whether I can reflectively get access. >>>> >>>> The rewritten method resolution would follow next, which would allow me to tackle that warning and >>>> see whether I can get rid of it. However, before going a wrong route I would like to learn what the >>>> "official" Java 9 solution would be and try to implement that. >>>> >>>> ---rony >>> Yes, I think you are dealing with two problems here which you have been using the same solution >>> for in the past. >>> >>> The 1st thing you have been doing incorrectly for Java 9, as Alan explained, is the idiom: >>> o.getClass().getMethod(...) and the 2nd is that you are trying to access protected members on >>> behalf of some other class which is a subclass of the protected member's declaring class. >>> >>> The 1st problem has different solutions which are all doable in Java 9, since you are dealing >>> within the confines of public types, public members and exported packages. One solution is to >>> search for the most specific member in the inheritance hierarchy which is also accessible >>> (declared in public type in exported package) which is what Alan suggests. >>> >>> There might also be another elegant solution which requires some re-design of your Rexx >>> interpreter.? When you deal with reference values in Rexx (the values that refer to objects in >>> Java), you could track not only the value itself but also the "static" type of that value. A >>> reference value is always obtained either by calling a constructor, accessing a field (either >>> static or instance), by calling a method (static or instance) or by accessing an element of some >>> array: >>> >>> - calling constructor: the "static type" is the class upon which the constructor has been called >>> - accessing a field: the "static type" is the type of the field (i.e. Field.getDeclaringClass()) >>> - calling a method: the "static type" is the return type of the method (i.e. Method.getReturnType()) >>> - accessing an element of some array: the "static type" is the array's "static type"'s component >>> type (i.e. Class.getComponentType() invoked on array's "static type" Class). >>> >>> When you take the "static" type as the starting Class when searching for a public member with >>> standard Class.getMethod() or Class.getField(), you would then get the correct publicly accessible >>> reflected member. With a caveat that this only works when there's no generics involved. If there's >>> generics, the logic to compute the correct static type is more involved and would sometimes >>> require passing the generic type parameters (when invoking constructors of generic classes or >>> generic methods) in the syntax of your Rexx language. So you may or may not want to do that. >>> Perhaps some library for deep resolving could be of help here (Google Guava has some support for >>> that). I guess searching for the most specific member in the hierarchy that is also accessible is >>> your best bet currently if the goal is to be syntactically backwards compatible in the Rexx language. >>> >>> The 2nd problem is not trivial as you want to access a protected member on behalf of some other >>> sub-class of the member's declaring class which is not cooperating (voluntarily handing you an >>> instance of its Lookup object). This currently requires the package containing the member's >>> declaring class to be opened at least to you (the Rexx interpreter) and using the >>> member.setAccessible(true) trick or MethodHandles.privateLookupIn(declaringClass) equivalent for >>> method handles. Which is awkward because libraries packed as modules would normally not specify >>> that in their module descriptors and system modules don't either. So you are left with either >>> --add-opens command line switches or deploying a javaagent to the JVM and using it's API point >>> java.lang.instrument.Instrumentation#redefineModule to add opens to modules that way. Both >>> approaches are not elegant, but that's what is currently available, I think. >>> >>> Regards, Peter > From Rony.Flatscher at wu.ac.at Tue Jan 23 14:52:46 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 23 Jan 2018 15:52:46 +0100 Subject: Reflection: how can one access public fields (members) in a superclass ? Message-ID: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> Given three modules (sources at the end) where * "mod_A" exports its package "mtest1", to everyone * "mod_B" requires "mod_A" and exports its package "mtest2" to "mod_C" * "mod_C" requires "mod_B" and exports its package "mtest3" to everyone "mod_B"'s class "mtest2.Class02A" defines two public fields, one static ("pubStaticFromClass02A") and one an instance ("pubFromClass02") one. Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, here the source: TestUse_mtest3_Class03A.java ??????? public class TestUse_mtest3_Class03A ??????? { ??????????? public static void main (String args[]) { ??????????????? mtest3.Class03A o=new mtest3.Class03A(); ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); ??????????? } ??????? } Compiling the above program and running it yields: o.pubStaticFromClass02A???? : static-mtest2.Class02A o.pubFromClass02A?????????? : instance-mtest2.Class02A o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], getMyClassName()=[class-mtest1.Class01A] Here is a 1:1 transcription from the above Java program to Rexx which uses Java reflection to achieve the same: test.rex ??????? o=.bsf~new("mtest3.Class03A")????????? -- create Java object ??????? say "o~pubStaticFromClass01A:" o~pubStaticFromClass02A ??????? say "o~pubFromClass01A????? :" o~pubFromClass02A ??????? say "o:" o "o~getMyClassName:" o~getMyClassName ??????? ::requires BSF.CLS?? -- direct interpreter to load Java bridge Running the Rexx program yields the following reflection error: // // -> -> RexxReflectJava9.processField(): EXCEPTION in GET-operation: tmpField="pubStaticFromClass02A" exception: "java.lang.IllegalAccessException: class org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access class mtest2.Class02A (in module mod_B) because module mod_B does not export mtest2? to unnamed module @51c8530f" The reflection code currently * gets the type from the Java object ("mtest3.Class03A") and tests whether the package "mtest3" is exported (it is), * looks for all declaredFields and finds none, so it gets the superclass "mtest2.Class02A", * looks for all declaredFields and locates the Field named "pubStaticFromClass02A" and invokes the Field's get method, supplying the Java object (an instance of class mtest3.Class03A) which causes an IlleagalAccessException. Although it is true that "mod_B" is not exported to the unnamed module it is still the case that "mod_C" is exported (and class "mtest3.Class03A" can be accessed), such that all public members in its superclasses should be accessible via reflection, even in the case that a public member resides in a module that is not exported to the reflector from the unnamed module? The reflective code would be able to assess that the supplied object is from an exported type and hence allow the get access in this case for reflected members in its superclasses, like it seems the Java compiler allows for. ---rony Here are the contents of the module directories in source: --------------------------------------------------------------------------- mod_A/module-info.java ??????? module mod_A { exports mtest1; } mod_A/mtest1/Class01A.java ??????? package mtest1; ??????? abstract public class Class01A ??????? { ??????????? protected static String myClassName = "class-mtest1.Class01A"; ??????? } --------------------------------------------------------------------------- mod_B/module-info.java ??????? module mod_B { ??????????? requires mod_A; ??????????? exports mtest2 to mod_C; ??????? } mod_B/mtest2/Class02A.java ??????? package mtest2; ??????? public class Class02A extends mtest1.Class01A ??????? { ??????????? public static String pubStaticFromClass02A="static-mtest2.Class02A"; ??????????? public??????? String pubFromClass02A????? ="instance-mtest2.Class02A"; ??????????? public String getMyClassName() ??????????? { ??????????????? return "via: this=["+this+"], getMyClassName()=["+myClassName+"]"; ??????????? } ??????? } --------------------------------------------------------------------------- mod_C/module-info.java ??????? module mod_B { ??????????? requires mod_A; ??????????? exports mtest2 to mod_C; ??????? } mod_C/mtest3/Class03A.java ??????? package mtest3; ??????? public class Class03A extends mtest2.Class02A ??????? { ??????? } -------------------------------------------------------------------------- From Rony.Flatscher at wu.ac.at Tue Jan 23 17:32:58 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 23 Jan 2018 18:32:58 +0100 Subject: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> Message-ID: <79708b38-12ef-eeae-86e5-78af1e3aa177@wu.ac.at> Oh, forgot the scripts to compile and run (these are under Windows): Assuming that the sources of "mod_A", "mod_B" and m"od_C" are located in "src", the compilation result should be placed into "out" here the compile script: rd out /s /q? && md out @echo compiling module mod_A dir????? src\mod_A\*java /s /b > mod_A_source_files.txt type mod_A_source_files.txt javac -d out\mod_A @mod_A_source_files.txt @echo compiling module mod_B dir????? src\mod_B\*java /s /b > mod_B_source_files.txt type mod_B_source_files.txt javac --module-path out -d out\mod_B @mod_B_source_files.txt @echo compiling module mod_C dir????? src\mod_C\*java /s /b > mod_C_source_files.txt type mod_C_source_files.txt javac --module-path out -d out\mod_C @mod_C_source_files.txt Assuming that "TestUse_mtest3_Class03A.java" is located in the directory that contains the "out" subdirectory: del TestUse_mtest3_Class03A.class javac -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A.java java? -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A ---rony On 23.01.2018 15:52, Rony G. Flatscher wrote: > Given three modules (sources at the end) where > > * "mod_A" exports its package "mtest1", to everyone > * "mod_B" requires "mod_A" and exports its package "mtest2" to "mod_C" > * "mod_C" requires "mod_B" and exports its package "mtest3" to everyone > > "mod_B"'s class "mtest2.Class02A" defines two public fields, one static ("pubStaticFromClass02A") > and one an instance ("pubFromClass02") one. > > Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, > here the source: > > TestUse_mtest3_Class03A.java > > ??????? public class TestUse_mtest3_Class03A > ??????? { > ??????????? public static void main (String args[]) { > ??????????????? mtest3.Class03A o=new mtest3.Class03A(); > ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); > ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); > ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); > ??????????? } > ??????? } > > Compiling the above program and running it yields: > > o.pubStaticFromClass02A???? : static-mtest2.Class02A > o.pubFromClass02A?????????? : instance-mtest2.Class02A > o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], > getMyClassName()=[class-mtest1.Class01A] > > Here is a 1:1 transcription from the above Java program to Rexx which uses Java reflection to > achieve the same: > > test.rex > > ??????? o=.bsf~new("mtest3.Class03A")????????? -- create Java object > ??????? say "o~pubStaticFromClass01A:" o~pubStaticFromClass02A > ??????? say "o~pubFromClass01A????? :" o~pubFromClass02A > ??????? say "o:" o "o~getMyClassName:" o~getMyClassName > > ??????? ::requires BSF.CLS?? -- direct interpreter to load Java bridge > > Running the Rexx program yields the following reflection error: > > // // -> -> RexxReflectJava9.processField(): EXCEPTION in GET-operation: > tmpField="pubStaticFromClass02A" exception: "java.lang.IllegalAccessException: class > org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access class mtest2.Class02A (in module > mod_B) because module mod_B does not export mtest2? to unnamed module @51c8530f" > > The reflection code currently > > * gets the type from the Java object ("mtest3.Class03A") and tests whether the package "mtest3" is > exported (it is), > * looks for all declaredFields and finds none, so it gets the superclass "mtest2.Class02A", > * looks for all declaredFields and locates the Field named "pubStaticFromClass02A" and invokes the > Field's get method, supplying the Java object (an instance of class mtest3.Class03A) which > causes an IlleagalAccessException. > > Although it is true that "mod_B" is not exported to the unnamed module it is still the case that > "mod_C" is exported (and class "mtest3.Class03A" can be accessed), such that all public members in > its superclasses should be accessible via reflection, even in the case that a public member resides > in a module that is not exported to the reflector from the unnamed module? > > The reflective code would be able to assess that the supplied object is from an exported type and > hence allow the get access in this case for reflected members in its superclasses, like it seems the > Java compiler allows for. > > ---rony > > Here are the contents of the module directories in source: > > --------------------------------------------------------------------------- > > mod_A/module-info.java > > ??????? module mod_A { exports mtest1; } > > mod_A/mtest1/Class01A.java > > ??????? package mtest1; > > ??????? abstract public class Class01A > ??????? { > ??????????? protected static String myClassName = "class-mtest1.Class01A"; > ??????? } > > --------------------------------------------------------------------------- > > mod_B/module-info.java > > ??????? module mod_B { > ??????????? requires mod_A; > ??????????? exports mtest2 to mod_C; > ??????? } > > mod_B/mtest2/Class02A.java > > ??????? package mtest2; > > ??????? public class Class02A extends mtest1.Class01A > ??????? { > ??????????? public static String pubStaticFromClass02A="static-mtest2.Class02A"; > ??????????? public??????? String pubFromClass02A????? ="instance-mtest2.Class02A"; > > ??????????? public String getMyClassName() > ??????????? { > ??????????????? return "via: this=["+this+"], getMyClassName()=["+myClassName+"]"; > ??????????? } > ??????? } > > --------------------------------------------------------------------------- > > mod_C/module-info.java > > ??????? module mod_B { > ??????????? requires mod_A; > ??????????? exports mtest2 to mod_C; > ??????? } > > mod_C/mtest3/Class03A.java > > ??????? package mtest3; > > ??????? public class Class03A extends mtest2.Class02A > ??????? { > ??????? } > > -------------------------------------------------------------------------- > > From Rony.Flatscher at wu.ac.at Tue Jan 23 18:17:47 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 23 Jan 2018 19:17:47 +0100 Subject: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <79708b38-12ef-eeae-86e5-78af1e3aa177@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <79708b38-12ef-eeae-86e5-78af1e3aa177@wu.ac.at> Message-ID: <338ea0de-db99-c055-8503-1ca1dfa45f36@wu.ac.at> Was asked off-line for a zip-archive for convenience, hence I created a zip archive and put it on Dropbox. Here the link to get the zip-archive: .? Just unzip, go into the <02-20180123-getPublicField> directory and run (on Windows) "1_compile.cmd", then either? "5a_module_compile_and_run_TestUse_mtest3_Class03A.cmd" or "5b_classpath_compile_and_run_TestUse_mtest3_Class03A.cmd". Adapting the scripts to Unix should be straight forward. Being formally an OpenJDK contributor there should be no legal problems. ---rony On 23.01.2018 18:32, Rony G. Flatscher wrote: > Oh, forgot the scripts to compile and run (these are under Windows): > > Assuming that the sources of "mod_A", "mod_B" and m"od_C" are located in "src", the compilation > result should be placed into "out" here the compile script: > > rd out /s /q? && md out > > @echo compiling module mod_A > dir????? src\mod_A\*java /s /b > mod_A_source_files.txt > type mod_A_source_files.txt > javac -d out\mod_A @mod_A_source_files.txt > > @echo compiling module mod_B > dir????? src\mod_B\*java /s /b > mod_B_source_files.txt > type mod_B_source_files.txt > javac --module-path out -d out\mod_B @mod_B_source_files.txt > > @echo compiling module mod_C > dir????? src\mod_C\*java /s /b > mod_C_source_files.txt > type mod_C_source_files.txt > javac --module-path out -d out\mod_C @mod_C_source_files.txt > > Assuming that "TestUse_mtest3_Class03A.java" is located in the directory that contains the "out" > subdirectory: > > del TestUse_mtest3_Class03A.class > javac -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A.java > > java? -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A > > ---rony > > > On 23.01.2018 15:52, Rony G. Flatscher wrote: >> Given three modules (sources at the end) where >> >> * "mod_A" exports its package "mtest1", to everyone >> * "mod_B" requires "mod_A" and exports its package "mtest2" to "mod_C" >> * "mod_C" requires "mod_B" and exports its package "mtest3" to everyone >> >> "mod_B"'s class "mtest2.Class02A" defines two public fields, one static ("pubStaticFromClass02A") >> and one an instance ("pubFromClass02") one. >> >> Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, >> here the source: >> >> TestUse_mtest3_Class03A.java >> >> ??????? public class TestUse_mtest3_Class03A >> ??????? { >> ??????????? public static void main (String args[]) { >> ??????????????? mtest3.Class03A o=new mtest3.Class03A(); >> ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); >> ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); >> ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); >> ??????????? } >> ??????? } >> >> Compiling the above program and running it yields: >> >> o.pubStaticFromClass02A???? : static-mtest2.Class02A >> o.pubFromClass02A?????????? : instance-mtest2.Class02A >> o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], >> getMyClassName()=[class-mtest1.Class01A] >> >> Here is a 1:1 transcription from the above Java program to Rexx which uses Java reflection to >> achieve the same: >> >> test.rex >> >> ??????? o=.bsf~new("mtest3.Class03A")????????? -- create Java object >> ??????? say "o~pubStaticFromClass01A:" o~pubStaticFromClass02A >> ??????? say "o~pubFromClass01A????? :" o~pubFromClass02A >> ??????? say "o:" o "o~getMyClassName:" o~getMyClassName >> >> ??????? ::requires BSF.CLS?? -- direct interpreter to load Java bridge >> >> Running the Rexx program yields the following reflection error: >> >> // // -> -> RexxReflectJava9.processField(): EXCEPTION in GET-operation: >> tmpField="pubStaticFromClass02A" exception: "java.lang.IllegalAccessException: class >> org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access class mtest2.Class02A (in module >> mod_B) because module mod_B does not export mtest2? to unnamed module @51c8530f" >> >> The reflection code currently >> >> * gets the type from the Java object ("mtest3.Class03A") and tests whether the package "mtest3" is >> exported (it is), >> * looks for all declaredFields and finds none, so it gets the superclass "mtest2.Class02A", >> * looks for all declaredFields and locates the Field named "pubStaticFromClass02A" and invokes the >> Field's get method, supplying the Java object (an instance of class mtest3.Class03A) which >> causes an IlleagalAccessException. >> >> Although it is true that "mod_B" is not exported to the unnamed module it is still the case that >> "mod_C" is exported (and class "mtest3.Class03A" can be accessed), such that all public members in >> its superclasses should be accessible via reflection, even in the case that a public member resides >> in a module that is not exported to the reflector from the unnamed module? >> >> The reflective code would be able to assess that the supplied object is from an exported type and >> hence allow the get access in this case for reflected members in its superclasses, like it seems the >> Java compiler allows for. >> >> ---rony >> >> Here are the contents of the module directories in source: >> >> --------------------------------------------------------------------------- >> >> mod_A/module-info.java >> >> ??????? module mod_A { exports mtest1; } >> >> mod_A/mtest1/Class01A.java >> >> ??????? package mtest1; >> >> ??????? abstract public class Class01A >> ??????? { >> ??????????? protected static String myClassName = "class-mtest1.Class01A"; >> ??????? } >> >> --------------------------------------------------------------------------- >> >> mod_B/module-info.java >> >> ??????? module mod_B { >> ??????????? requires mod_A; >> ??????????? exports mtest2 to mod_C; >> ??????? } >> >> mod_B/mtest2/Class02A.java >> >> ??????? package mtest2; >> >> ??????? public class Class02A extends mtest1.Class01A >> ??????? { >> ??????????? public static String pubStaticFromClass02A="static-mtest2.Class02A"; >> ??????????? public??????? String pubFromClass02A????? ="instance-mtest2.Class02A"; >> >> ??????????? public String getMyClassName() >> ??????????? { >> ??????????????? return "via: this=["+this+"], getMyClassName()=["+myClassName+"]"; >> ??????????? } >> ??????? } >> >> --------------------------------------------------------------------------- >> >> mod_C/module-info.java >> >> ??????? module mod_B { >> ??????????? requires mod_A; >> ??????????? exports mtest2 to mod_C; >> ??????? } >> >> mod_C/mtest3/Class03A.java >> >> ??????? package mtest3; >> >> ??????? public class Class03A extends mtest2.Class02A >> ??????? { >> ??????? } >> >> -------------------------------------------------------------------------- From Rony.Flatscher at wu.ac.at Wed Jan 24 10:29:58 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 24 Jan 2018 11:29:58 +0100 Subject: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <338ea0de-db99-c055-8503-1ca1dfa45f36@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <79708b38-12ef-eeae-86e5-78af1e3aa177@wu.ac.at> <338ea0de-db99-c055-8503-1ca1dfa45f36@wu.ac.at> Message-ID: <84a3b3eb-9e92-3640-ee02-5c53d8f04646@wu.ac.at> As it does not make sense to supply my current "under construction" code, I created two Java programs, one using reflection and one MethodHandles that behave like my current testbed. I added those two Java programs "TestUseViaReflection.java" (compile and run with "7a_module_compile_and_run_TestUseViaReflection.cmd") and "TestUseViaReflectionAndMethodHandles.java" (compile and run with "8a_module_compile_and_run_TestUseViaReflectionAndMethodHandles.cmd") to the zip archive and replaced the zip-archive on Dropbox () with the updated one. This is the "pure reflection" version, that I would prefer, "TestUseViaReflection.java": import java.lang.reflect.Field; public class TestUseViaReflection { ??? public static void main (String args[]) { ??????? mtest3.Class03A o=new mtest3.Class03A(); ??????? Class sc???? =o.getClass().getSuperclass(); // get superclass mtest2.Class02A ??????? Field? field = null; ??????? try { ??????????? field =sc.getDeclaredField("pubStaticFromClass02A");? // get static field ??????? } ??????? catch (Exception e) ??????? { ??????????? System.err.println("exception thrown in sc.getDeclaredField(...): "+e); ??????????? e.printStackTrace(); ??????????? System.exit(-1); ??????? } ??????? Object result=null; ??????? try { ??????????? result=field.get(o);????????????????? // get field's value ??????? } ??????? catch (Exception e) ??????? { ??????????? System.err.println("exception thrown in field.get(...): "+e); ??????????? e.printStackTrace(); ??????????? System.exit(-1); ??????? } ??????? System.out.println("o.pubStaticFromClass02A???? : "+result ); ??? } } ?Running it yields: G:\xfer\java9modules\02-20180123-getPublicField>java? -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUseViaReflection exception thrown in field.get(...): java.lang.IllegalAccessException: class TestUseViaReflection cannot access class mtest2.Class02A (in module mod_B) ?because module mod_B does not export mtest2 to unnamed module @4e04a765 java.lang.IllegalAccessException: class TestUseViaReflection cannot access class mtest2.Class02A (in module mod_B) because module mod_B does not export mtest2 to unnamed module @4e04a765 ??????? at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361) ??????? at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:589) ??????? at java.base/java.lang.reflect.Field.checkAccess(Field.java:1075) ??????? at java.base/java.lang.reflect.Field.get(Field.java:416) ??????? at TestUseViaReflection.main(TestUseViaReflection.java:23) ---------------- This is the version that tries to use MethodHandles instead, "TestUseViaReflectionAndMethodHandles.java": import java.lang.reflect.Field; import java.lang.invoke.*; public class TestUseViaReflectionAndMethodHandles { ??? public static MethodHandles.Lookup lookup=MethodHandles.lookup(); ??? public static void main (String args[]) { ??????? mtest3.Class03A o=new mtest3.Class03A(); ??????? Class sc???? =o.getClass().getSuperclass(); // get superclass mtest2.Class02A ??????? Field? field = null; ??????? try { ??????????? field =sc.getDeclaredField("pubStaticFromClass02A");? // get static field ??????? } ??????? catch (Exception e) ??????? { ??????????? System.err.println("--> exception thrown in sc.getDeclaredField(...): "+e); ??????????? e.printStackTrace(); ??????????? System.exit(-1); ??????? } ??????? Object result=null; ??????? MethodHandle mh; ??????? try { ??????????? mh=lookup.unreflectGetter(field); ??????????? result=mh.invoke(o); ??????? } ??????? catch (Throwable t) ??????? { ??????????? System.err.println("--> exception thrown in field.get(...): "+t); ??????????? t.printStackTrace(); ??????????? System.exit(-1); ??????? } ??????? System.out.println("o.pubStaticFromClass02A???? : "+result ); ??? } } ?Running it yields: G:\xfer\java9modules\02-20180123-getPublicField>java? -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUseViaReflectionAndMethodHandles --> exception thrown in field.get(...): java.lang.IllegalAccessException: access to public member failed: mtest2.Class02A.pubStaticFromClass02A/java.lang.String/getStatic, from TestUseViaReflectionAndMethodHandles (unnamed module @4e04a765) java.lang.IllegalAccessException: access to public member failed: mtest2.Class02A.pubStaticFromClass02A/java.lang.String/getStatic, from TestUseViaReflectionAndMethodHandles (unnamed module @4e04a765) ??????? at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:914) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.checkAccess(MethodHandles.java:2193) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.checkField(MethodHandles.java:2143) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.getDirectFieldCommon(MethodHandles.java:2355) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.getDirectFieldNoSecurityManager(MethodHandles.java:2350) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.unreflectField(MethodHandles.java:1863) ??????? at java.base/java.lang.invoke.MethodHandles$Lookup.unreflectGetter(MethodHandles.java:1854) ??????? at TestUseViaReflectionAndMethodHandles.main(TestUseViaReflectionAndMethodHandles.java:27) ---------------- However the Java program "TestUse_mtest3_Class03A.java" compiles and runs successfully: Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, here the source: TestUse_mtest3_Class03A.java ??????? public class TestUse_mtest3_Class03A ??????? { ??????????? public static void main (String args[]) { ??????????????? mtest3.Class03A o=new mtest3.Class03A(); ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); ??????????? } ??????? } Compiling the above program and running it yields: o.pubStaticFromClass02A???? : static-mtest2.Class02A o.pubFromClass02A?????????? : instance-mtest2.Class02A o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], getMyClassName()=[class-mtest1.Class01A] ---------------- Maybe I miss something obvious. ---rony On 23.01.2018 19:17, Rony G. Flatscher wrote: > Was asked off-line for a zip-archive for convenience, hence I created a zip archive and put it on > Dropbox. Here the link to get the zip-archive: > .? Just unzip, go > into the <02-20180123-getPublicField> directory and run (on Windows) "1_compile.cmd", then either? > "5a_module_compile_and_run_TestUse_mtest3_Class03A.cmd" or > "5b_classpath_compile_and_run_TestUse_mtest3_Class03A.cmd". Adapting the scripts to Unix should be > straight forward. > > Being formally an OpenJDK contributor there should be no legal problems. > > ---rony > > > On 23.01.2018 18:32, Rony G. Flatscher wrote: >> Oh, forgot the scripts to compile and run (these are under Windows): >> >> Assuming that the sources of "mod_A", "mod_B" and m"od_C" are located in "src", the compilation >> result should be placed into "out" here the compile script: >> >> rd out /s /q? && md out >> >> @echo compiling module mod_A >> dir????? src\mod_A\*java /s /b > mod_A_source_files.txt >> type mod_A_source_files.txt >> javac -d out\mod_A @mod_A_source_files.txt >> >> @echo compiling module mod_B >> dir????? src\mod_B\*java /s /b > mod_B_source_files.txt >> type mod_B_source_files.txt >> javac --module-path out -d out\mod_B @mod_B_source_files.txt >> >> @echo compiling module mod_C >> dir????? src\mod_C\*java /s /b > mod_C_source_files.txt >> type mod_C_source_files.txt >> javac --module-path out -d out\mod_C @mod_C_source_files.txt >> >> Assuming that "TestUse_mtest3_Class03A.java" is located in the directory that contains the "out" >> subdirectory: >> >> del TestUse_mtest3_Class03A.class >> javac -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A.java >> >> java? -cp "." --module-path out --add-modules mod_A,mod_B,mod_C TestUse_mtest3_Class03A >> >> ---rony >> >> >> On 23.01.2018 15:52, Rony G. Flatscher wrote: >>> Given three modules (sources at the end) where >>> >>> * "mod_A" exports its package "mtest1", to everyone >>> * "mod_B" requires "mod_A" and exports its package "mtest2" to "mod_C" >>> * "mod_C" requires "mod_B" and exports its package "mtest3" to everyone >>> >>> "mod_B"'s class "mtest2.Class02A" defines two public fields, one static ("pubStaticFromClass02A") >>> and one an instance ("pubFromClass02") one. >>> >>> Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, >>> here the source: >>> >>> TestUse_mtest3_Class03A.java >>> >>> ??????? public class TestUse_mtest3_Class03A >>> ??????? { >>> ??????????? public static void main (String args[]) { >>> ??????????????? mtest3.Class03A o=new mtest3.Class03A(); >>> ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); >>> ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); >>> ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); >>> ??????????? } >>> ??????? } >>> >>> Compiling the above program and running it yields: >>> >>> o.pubStaticFromClass02A???? : static-mtest2.Class02A >>> o.pubFromClass02A?????????? : instance-mtest2.Class02A >>> o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], >>> getMyClassName()=[class-mtest1.Class01A] >>> >>> Here is a 1:1 transcription from the above Java program to Rexx which uses Java reflection to >>> achieve the same: >>> >>> test.rex >>> >>> ??????? o=.bsf~new("mtest3.Class03A")????????? -- create Java object >>> ??????? say "o~pubStaticFromClass01A:" o~pubStaticFromClass02A >>> ??????? say "o~pubFromClass01A????? :" o~pubFromClass02A >>> ??????? say "o:" o "o~getMyClassName:" o~getMyClassName >>> >>> ??????? ::requires BSF.CLS?? -- direct interpreter to load Java bridge >>> >>> Running the Rexx program yields the following reflection error: >>> >>> // // -> -> RexxReflectJava9.processField(): EXCEPTION in GET-operation: >>> tmpField="pubStaticFromClass02A" exception: "java.lang.IllegalAccessException: class >>> org.rexxla.bsf.engines.rexx.RexxReflectJava9 cannot access class mtest2.Class02A (in module >>> mod_B) because module mod_B does not export mtest2? to unnamed module @51c8530f" >>> >>> The reflection code currently >>> >>> * gets the type from the Java object ("mtest3.Class03A") and tests whether the package "mtest3" is >>> exported (it is), >>> * looks for all declaredFields and finds none, so it gets the superclass "mtest2.Class02A", >>> * looks for all declaredFields and locates the Field named "pubStaticFromClass02A" and invokes the >>> Field's get method, supplying the Java object (an instance of class mtest3.Class03A) which >>> causes an IlleagalAccessException. >>> >>> Although it is true that "mod_B" is not exported to the unnamed module it is still the case that >>> "mod_C" is exported (and class "mtest3.Class03A" can be accessed), such that all public members in >>> its superclasses should be accessible via reflection, even in the case that a public member resides >>> in a module that is not exported to the reflector from the unnamed module? >>> >>> The reflective code would be able to assess that the supplied object is from an exported type and >>> hence allow the get access in this case for reflected members in its superclasses, like it seems the >>> Java compiler allows for. >>> >>> ---rony >>> >>> Here are the contents of the module directories in source: >>> >>> --------------------------------------------------------------------------- >>> >>> mod_A/module-info.java >>> >>> ??????? module mod_A { exports mtest1; } >>> >>> mod_A/mtest1/Class01A.java >>> >>> ??????? package mtest1; >>> >>> ??????? abstract public class Class01A >>> ??????? { >>> ??????????? protected static String myClassName = "class-mtest1.Class01A"; >>> ??????? } >>> >>> --------------------------------------------------------------------------- >>> >>> mod_B/module-info.java >>> >>> ??????? module mod_B { >>> ??????????? requires mod_A; >>> ??????????? exports mtest2 to mod_C; >>> ??????? } >>> >>> mod_B/mtest2/Class02A.java >>> >>> ??????? package mtest2; >>> >>> ??????? public class Class02A extends mtest1.Class01A >>> ??????? { >>> ??????????? public static String pubStaticFromClass02A="static-mtest2.Class02A"; >>> ??????????? public??????? String pubFromClass02A????? ="instance-mtest2.Class02A"; >>> >>> ??????????? public String getMyClassName() >>> ??????????? { >>> ??????????????? return "via: this=["+this+"], getMyClassName()=["+myClassName+"]"; >>> ??????????? } >>> ??????? } >>> >>> --------------------------------------------------------------------------- >>> >>> mod_C/module-info.java >>> >>> ??????? module mod_B { >>> ??????????? requires mod_A; >>> ??????????? exports mtest2 to mod_C; >>> ??????? } >>> >>> mod_C/mtest3/Class03A.java >>> >>> ??????? package mtest3; >>> >>> ??????? public class Class03A extends mtest2.Class02A >>> ??????? { >>> ??????? } >>> >>> -------------------------------------------------------------------------- From Alan.Bateman at oracle.com Wed Jan 24 14:28:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Jan 2018 14:28:14 +0000 Subject: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> Message-ID: On 23/01/2018 14:52, Rony G. Flatscher wrote: > Given three modules (sources at the end) where > > * "mod_A" exports its package "mtest1", to everyone > * "mod_B" requires "mod_A" and exports its package "mtest2" to "mod_C" > * "mod_C" requires "mod_B" and exports its package "mtest3" to everyone > > "mod_B"'s class "mtest2.Class02A" defines two public fields, one static ("pubStaticFromClass02A") > and one an instance ("pubFromClass02") one. > > Compiling the modules and then using them in the following Java program (via the CLASSPATH) works, > here the source: > > TestUse_mtest3_Class03A.java > > ??????? public class TestUse_mtest3_Class03A > ??????? { > ??????????? public static void main (String args[]) { > ??????????????? mtest3.Class03A o=new mtest3.Class03A(); > ??????????????? System.out.println("o.pubStaticFromClass02A???? : "+o.pubStaticFromClass02A ); > ??????????????? System.out.println("o.pubFromClass02A?????????? : "+o.pubFromClass02A???? ); > ??????????????? System.out.println("o: "+o+", o.getMyClassName(): "+o.getMyClassName()); > ??????????? } > ??????? } > > Compiling the above program and running it yields: > > o.pubStaticFromClass02A???? : static-mtest2.Class02A > o.pubFromClass02A?????????? : instance-mtest2.Class02A > o: mtest3.Class03A at 5afa04c, o.getMyClassName(): via: this=[mtest3.Class03A at 5afa04c], > getMyClassName()=[class-mtest1.Class01A] > I don't think the questions and observations in this thread are strictly modules related. One suggestion is to start with a simpler scenario like this: package p; class C1 { ??? public static final int K = 99; ??? public static int k() { return K; } ??? public final int F = -1; ??? public int m() { return F; } } package p; public class C2 extends C1 { } No modules or qualified exports in the picture for now. The important part is that C1 is not public but it has public members. You can try tests to see if references to C2.K, C2.k(), new C2().F, and new C2().m() will compile and run. You can try the equivalent with core reflection to see how it differs to static references (you may have to change method m to be final to prevent javac generating a bridge method in C2). -Alan. From Rony.Flatscher at wu.ac.at Wed Jan 24 15:42:11 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 24 Jan 2018 16:42:11 +0100 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> Message-ID: <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> Changed the subject to hint at the "p" package example. On 24.01.2018 15:28, Alan Bateman wrote: > On 23/01/2018 14:52, Rony G. Flatscher wrote: ... cut ... ? > I don't think the questions and observations in this thread are strictly modules related. One > suggestion is to start with a simpler scenario like this: > > package p; > class C1 { > ??? public static final int K = 99; > ??? public static int k() { return K; } > ??? public final int F = -1; > ??? public int m() { return F; } > } > > package p; > public class C2 extends C1 { } > > No modules or qualified exports in the picture for now. The important part is that C1 is not > public but it has public members. You can try tests to see if references to C2.K, C2.k(), new > C2().F, and new C2().m() will compile and run. You can try the equivalent with core reflection to > see how it differs to static references (you may have to change method m to be final to prevent > javac generating a bridge method in C2). OK, now add to this the following class that uses p.C2 objects to access e.g. m() via it: G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java public class UseC2 { ??? public static void main (String args[]) { ??????? p.C2 o=new p.C2(); ??????? System.out.println("o="+o); ??????? System.out.println("o.m()="+o.m()); ??? } } Compiling all three classes works. Running "UseC2" works and outputs: G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 o=p.C2 at 66048bfd o.m()=-1 So it is possible to access m() via the p.C2 object from UseC2. --- The modules seem to come into play when reflection or unreflect try to determine whether accessibility should be granted or not. The current implementations (mistakingly, I think) assume that access is only to be grantable if the package of the object to reflect is exported to the reflector. Rather, it should check whether the reflected member is in a class with a package that gets exported to the reflector *or* is a superclass of a class which package got exported to the reflector. Or with other words, once a class is determined that gets exported to the reflector all public members in all superclasses should be accessible in order to avoid "crippled Java objects CJO" ;) . java.lang.reflect.* could do that check. In the case of unreflection probably one should be able to supply an exported class to check accessibility and then only accept object that are instances of at least that class. ---rony From Rony.Flatscher at wu.ac.at Wed Jan 24 18:11:29 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 24 Jan 2018 19:11:29 +0100 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> Message-ID: <8bd36636-c88d-6c03-65c1-9482212808a2@wu.ac.at> Had to leave, hence continuing here. The purpose of demonstrating that compilable and runnable Java program is simply to proof, that it is legal to access public members of package private classes, if the access occurs via an instance of a public subclass. The reflective version that behaves like the presented compiled version, may be coded like: import java.lang.reflect.*; public class UseC2Reflective { ??? public static void main (String args[]) { ??????? p.C2 o=new p.C2(); ??????? System.out.println("o="+o); ??????? Class sc=o.getClass().getSuperclass(); ??????? Method rm??? =null; ??????? Object result=null; ??????? try { ??????????? rm=sc.getDeclaredMethod("m",new Class[0]); ??????????? try { ??????????????? result=rm.invoke(o,new Object[0]); ??????????? } ??????????? catch (Exception e1) ??????????? { ??????????????? rm.setAccessible(true); ??????????????? result=rm.invoke(o,new Object[0]); ??????????? } ??????? } ??????? catch (Exception e) ??????? { ??????????? System.err.println("getDefinedMethod(...) caused exception: "+e); ??????????? e.printStackTrace(); ??????????? System.exit(-1); ??????? } ??????? System.out.println("o.m()="+o.m()); ??? } } Running it with Java 9 gives: G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2Reflective o=p.C2 at 3cd1f1c8 o.m()=-1 So this behaves like in the past: it is possible to mimickry a compiled Java program using reflection with setAccessible. ----------------- Now turning to the use of setAccessible() and looking up the Javadocs for Java 6, 7, 8 and 9, is interesting: * Javadocs 6: public class AccessibleObject extends Object implements AnnotatedElement The AccessibleObject class is the base class for Field, Method and Constructor objects. It provides the ability to flag a reflected object as suppressing default Java language access control checks when it is used. The access checks--for public, default (package) access, protected, and private members--are performed when Fields, Methods or Constructors are used to set or get fields, to invoke methods, or to create and initialize new instances of classes, respectively. Setting the accessible flag in a reflected object permits sophisticated applications with sufficient privilege, such as Java Object Serialization or other persistence mechanisms, to manipulate objects in a manner that would normally be prohibited. Since: ??? 1.2 See Also: ??? Field, Method, Constructor, ReflectPermission * Javadocs 7; this adds a paragraph to the documentation: public class AccessibleObject extends Object implements AnnotatedElement The AccessibleObject class is the base class for Field, Method and Constructor objects. It provides the ability to flag a reflected object as suppressing default Java language access control checks when it is used. The access checks--for public, default (package) access, protected, and private members--are performed when Fields, Methods or Constructors are used to set or get fields, to invoke methods, or to create and initialize new instances of classes, respectively. Setting the accessible flag in a reflected object permits sophisticated applications with sufficient privilege, such as Java Object Serialization or other persistence mechanisms, to manipulate objects in a manner that would normally be prohibited. *By default, a reflected object is not accessible.* * Javadocs 8; this is the same as for the Javadocs 7. So the rule is, that a reflected object is not accessible by default, such that one needs to use the setAccessible() method. * Javadocs 9; this rewrites the documentation to read: public class AccessibleObject extends Object implements AnnotatedElement The AccessibleObject class is the base class for Field, Method, and Constructor objects (known as reflected objects). It provides the ability to flag a reflected object as suppressing checks for Java language access control when it is used. This permits sophisticated applications with sufficient privilege, such as Java Object Serialization or other persistence mechanisms, to manipulate objects in a manner that would normally be prohibited. Java language access control prevents use of private members outside their class; package access members outside their package; protected members outside their package or subclasses; and public members outside their module unless they are declared in an exported package and the user reads their module. By default, Java language access control is enforced (with one variation) when Fields, Methods, or Constructors are used to get or set fields, to invoke methods, or to create and initialize new instances of classes, respectively. Every reflected object checks that the code using it is in an appropriate class, package, or module. The one variation from Java language access control is that the checks by reflected objects assume readability. That is, the module containing the use of a reflected object is assumed to read the module in which the underlying field, method, or constructor is declared. Whether the checks for Java language access control can be suppressed (and thus, whether access can be enabled) depends on whether the reflected object corresponds to a member in an exported or open package (see setAccessible(boolean)). Since: ??? 1.2 All of a sudden it states that "access control prevents use of (...) package access members outside their package (...)" by the "Java language access control", however the compiled Java program is allowed to do so! I think it makes perfect sense that if a public subclass having package access and returning an object that the receiver of that object is rightfully entitled to use its public members available in its class and all of its superclasses. There is no risk that the one who has a reference to such an object can do any harm, if using public members. (As package classes have always access to their peer package classes and access to their peer's package members it would probably not make sense to allow "public" in that context, if access to those members was not allowed to become "public" outside of the package eventually.) For the module system it states "(...) prevents (...) public members outside their module unless they are declared in an exported package and the user reads their module". I think this is not specified enough. Using the same arguments as in the above paragraph: once an object is handed out, and if its class or one of its superclasses got exported, then starting with the exported class all public members of that class and all of its superclasses should be accessible via reflection as well. There is no risk that the one who has a reference to such an object can do any harm to the module system, if being restricted of using public members starting with the first exported class and then all of its superclasses as well. ---rony On 24.01.2018 16:42, Rony G. Flatscher wrote: > Changed the subject to hint at the "p" package example. > > > > On 24.01.2018 15:28, Alan Bateman wrote: >> On 23/01/2018 14:52, Rony G. Flatscher wrote: > ... cut ... > > ? >> I don't think the questions and observations in this thread are strictly modules related. One >> suggestion is to start with a simpler scenario like this: >> >> package p; >> class C1 { >> ??? public static final int K = 99; >> ??? public static int k() { return K; } >> ??? public final int F = -1; >> ??? public int m() { return F; } >> } >> >> package p; >> public class C2 extends C1 { } >> >> No modules or qualified exports in the picture for now. The important part is that C1 is not >> public but it has public members. You can try tests to see if references to C2.K, C2.k(), new >> C2().F, and new C2().m() will compile and run. You can try the equivalent with core reflection to >> see how it differs to static references (you may have to change method m to be final to prevent >> javac generating a bridge method in C2). > OK, now add to this the following class that uses p.C2 objects to access e.g. m() via it: > > G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java > > public class UseC2 > { > ??? public static void main (String args[]) { > ??????? p.C2 o=new p.C2(); > ??????? System.out.println("o="+o); > ??????? System.out.println("o.m()="+o.m()); > ??? } > } > > Compiling all three classes works. > > Running "UseC2" works and outputs: > > G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 > o=p.C2 at 66048bfd > o.m()=-1 > > So it is possible to access m() via the p.C2 object from UseC2. > > --- > > The modules seem to come into play when reflection or unreflect try to determine whether > accessibility should be granted or not. The current implementations (mistakingly, I think) assume > that access is only to be grantable if the package of the object to reflect is exported to the > reflector. > > Rather, it should check whether the reflected member is in a class with a package that gets exported > to the reflector *or* is a superclass of a class which package got exported to the reflector. Or > with other words, once a class is determined that gets exported to the reflector all public members > in all superclasses should be accessible in order to avoid "crippled Java objects CJO" ;) . > > java.lang.reflect.* could do that check. In the case of unreflection probably one should be able to > supply an exported class to check accessibility and then only accept object that are instances of at > least that class. > > ---rony > > > > From Rony.Flatscher at wu.ac.at Wed Jan 24 18:35:50 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Wed, 24 Jan 2018 19:35:50 +0100 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <8bd36636-c88d-6c03-65c1-9482212808a2@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> <8bd36636-c88d-6c03-65c1-9482212808a2@wu.ac.at> Message-ID: <0729c22a-4168-ba72-a3ca-b7e8549388c0@wu.ac.at> On 24.01.2018 19:11, Rony G. Flatscher wrote: ... cut ... > So the rule is, that a reflected object is not accessible by default, such that one needs to use the > setAccessible() method. > > * Javadocs 9; this rewrites the documentation to read: > > public class AccessibleObject > extends Object > implements AnnotatedElement > > The AccessibleObject class is the base class for Field, Method, and Constructor objects (known > as reflected objects). It provides the ability to flag a reflected object as suppressing checks > for Java language access control when it is used. This permits sophisticated applications with > sufficient privilege, such as Java Object Serialization or other persistence mechanisms, to > manipulate objects in a manner that would normally be prohibited. > > Java language access control prevents use of private members outside their class; package access > members outside their package; protected members outside their package or subclasses; and public > members outside their module unless they are declared in an exported package and the user reads > their module. By default, Java language access control is enforced (with one variation) when > Fields, Methods, or Constructors are used to get or set fields, to invoke methods, or to create > and initialize new instances of classes, respectively. Every reflected object checks that the > code using it is in an appropriate class, package, or module. > > The one variation from Java language access control is that the checks by reflected objects > assume readability. That is, the module containing the use of a reflected object is assumed to > read the module in which the underlying field, method, or constructor is declared. > > Whether the checks for Java language access control can be suppressed (and thus, whether access > can be enabled) depends on whether the reflected object corresponds to a member in an exported > or open package (see setAccessible(boolean)). > > Since: > ??? 1.2 > > All of a sudden it states that "access control prevents use of (...) package access members outside > their package (...)" by the "Java language access control", however the compiled Java program is > allowed to do so! Sorry, misread that part in my rush (short break between two short meetings): it is of course o.k. to not allow access to package access members outside their package or subclasses! There is no general rule here that would prevent access to public access members, except for the new Module system. > I think it makes perfect sense that if a public subclass having package access and > returning an object that the receiver of that object is rightfully entitled to use its public > members available in its class and all of its superclasses. There is no risk that the one who has a > reference to such an object can do any harm, if using public members. (As package classes have > always access to their peer package classes and access to their peer's package members it would > probably not make sense to allow "public" in that context, if access to those members was not > allowed to become "public" outside of the package eventually.) > > For the module system it states "(...) prevents (...) public members outside their module unless > they are declared in an exported package and the user reads their module". I think this is not > specified enough. Using the same arguments as in the above paragraph: once an object is handed out, > and if its class or one of its superclasses got exported, then starting with the exported class all > public members of that class and all of its superclasses should be accessible via reflection as > well. There is no risk that the one who has a reference to such an object can do any harm to the > module system, if being restricted of using public members starting with the first exported class > and then all of its superclasses as well. > > ---rony > ... cut ... ---rony From Alan.Bateman at oracle.com Wed Jan 24 19:48:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Jan 2018 19:48:31 +0000 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> Message-ID: <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> On 24/01/2018 15:42, Rony G. Flatscher wrote: > > OK, now add to this the following class that uses p.C2 objects to > access e.g. m() via it: > > G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java > > public class UseC2 > { > ??? public static void main (String args[]) { > ??????? p.C2 o=new p.C2(); > ??????? System.out.println("o="+o); > ??????? System.out.println("o.m()="+o.m()); > ??? } > } > > Compiling all three classes works. > > Running "UseC2" works and outputs: > > G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 > o=p.C2 at 66048bfd > o.m()=-1 > > So it is possible to access m() via the p.C2 object from UseC2. That's right and this is the reason for moving to the simpler example. To be absolutely sure then you should decompile C2.class to make sure that there isn't a bridge method calling C1's m2(). If you change m() to be final then that will keep the bridge method from complicating the picture. If you change UseC2 to use core reflection and you hit the issue because the Method object you get is p.C1.m(). Attempting to invoke this will fail with IllegalAccessException. In your other mail you show a code fragment where it catches exceptions and calls setAccessible - I'll guess that this may have been masking the issue in the Rexx bridge. For completeness then you may want to try out the new reflection API. I realize you have to compile to JDK 6 but I think you'll find it will work the same way as the invokevirtual that o.m() compiles to. ??? MethodHandles.Lookup lookup = MethodHandles.lookup(); ??? MethodType mt = MethodType.methodType(int.class); ??? MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt); ??? Object res = mh.invoke(o); -Alan From jeffrey_kutcher at yahoo.com Thu Jan 25 21:54:31 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Thu, 25 Jan 2018 21:54:31 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <806d80bb-74c5-c12b-b550-dd8008cbf219@gmx.org> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> <1401341799.487077.1516283966017@mail.yahoo.com> <806d80bb-74c5-c12b-b550-dd8008cbf219@gmx.org> Message-ID: <1596534197.549113.1516917271709@mail.yahoo.com> VBox class hierarchy shows getChildren() are implemented in Parentand Pane. vbox.getChildren() returns an instance of Parent which isprotected rather than public found in Pane. Accessing the Listreturned by Parent is an illegal reference because it is protectedwhile accessing the List returned by Pane should be legal, butthis isn't whats returned. VBox Class Hierarchy: java.lang.Objectjavafx.scene.Nodejavafx.scene.Parent -> protected ObservableList getChildren?()javafx.scene.layout.Regionjavafx.scene.layout.Pane -> public ObservableList getChildren()javafx.scene.layout.VBox Class instance of vbox.getChildren() is: vbox.getChildren().getClass() -> class javafx.scene.Parent$3 Pane is widening the access restrictions of Parent. Is this intentional?Does it have any effect? Should Parent's access restrictions of getChildren()be public also or should Pane's be protected? My thought is public for both. From jeffrey_kutcher at yahoo.com Fri Jan 26 16:40:15 2018 From: jeffrey_kutcher at yahoo.com (jeffrey kutcher) Date: Fri, 26 Jan 2018 16:40:15 +0000 (UTC) Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1596534197.549113.1516917271709@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> <1401341799.487077.1516283966017@mail.yahoo.com> <806d80bb-74c5-c12b-b550-dd8008cbf219@gmx.org> <1596534197.549113.1516917271709@mail.yahoo.com> Message-ID: <1418317287.984683.1516984815771@mail.yahoo.com> VBox class hierarchy shows getChildren() is implemented in Parent and Pane. vbox.getChildren() returns an instance of Parent which is protected rather than public found in Pane. Accessing the List returned by Parent is an illegal reference because it is protected while accessing the List returned by Pane should be legal, but this isn't whats returned. VBox Class Hierarchy: java.lang.Object javafx.scene.Node javafx.scene.Parent -> protected ObservableList getChildren?() javafx.scene.layout.Region javafx.scene.layout.Pane -> public ObservableList getChildren() javafx.scene.layout.VBox Class instance of vbox.getChildren() is: vbox.getChildren().getClass() -> class javafx.scene.Parent$3 Pane is widening the access restrictions of Parent. Is this intentional? Does it have any effect? Should Parent's access restrictions of getChildren() be public or should Pane's be protected? My thought is public for both. From kevin.rushforth at oracle.com Fri Jan 26 18:14:48 2018 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Fri, 26 Jan 2018 10:14:48 -0800 Subject: How does one handle a java method which returns a non-public sub-type via reflection? In-Reply-To: <1418317287.984683.1516984815771@mail.yahoo.com> References: <974594798.1859150.1515765646023.ref@mail.yahoo.com> <974594798.1859150.1515765646023@mail.yahoo.com> <519854177.2043534.1515781595149@mail.yahoo.com> <49301555-420b-5800-ce31-4e5ca49e9bd2@oracle.com> <412824504.2154912.1515791840103@mail.yahoo.com> <1a095b48-6c24-3716-b34e-f7c07c7588f4@poczta.onet.pl> <1152555021.3613809.1516040839387@mail.yahoo.com> <474aea2c-b084-ebd3-4456-07507201e5b0@oracle.com> <1677507611.4168582.1516108118496@mail.yahoo.com> <1761664711.4166054.1516111069246@mail.yahoo.com> <5A5E0AEC.3080906@oracle.com> <1401341799.487077.1516283966017@mail.yahoo.com> <806d80bb-74c5-c12b-b550-dd8008cbf219@gmx.org> <1596534197.549113.1516917271709@mail.yahoo.com> <1418317287.984683.1516984815771@mail.yahoo.com> Message-ID: <5A6B7018.4040403@oracle.com> jeffrey kutcher wrote: > > VBox class hierarchy shows getChildren() is implemented in Parent > and Pane. vbox.getChildren() returns an instance of Parent which is > protected rather than public found in Pane. Accessing the List > returned by Parent is an illegal reference because it is protected > while accessing the List returned by Pane should be legal, but > this isn't whats returned. > > VBox Class Hierarchy: > > java.lang.Object > javafx.scene.Node > javafx.scene.Parent -> protected ObservableList getChildren?() > javafx.scene.layout.Region > javafx.scene.layout.Pane -> public ObservableList getChildren() > javafx.scene.layout.VBox > > Class instance of vbox.getChildren() is: > > vbox.getChildren().getClass() -> class javafx.scene.Parent$3 > > Pane is widening the access restrictions of Parent. Is this intentional? > Does it have any effect? Should Parent's access restrictions of getChildren() > be public or should Pane's be protected? > > My thought is public for both. > Yes, this is intentional. It is fine for a subclass to widen the access from protected to public. No, we don't want to make Parent::getChildren public, since there are some subclasses that do not want to expose direct access to application. -- Kevin From Rony.Flatscher at wu.ac.at Sat Jan 27 18:26:25 2018 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Sat, 27 Jan 2018 19:26:25 +0100 Subject: p example (Re: Reflection: how can one access public fields (members) in a superclass ? In-Reply-To: <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> References: <5c7fdab3-596f-8de9-49d0-a440e215f2cb@wu.ac.at> <9b26b3b8-7c5e-f0ab-6742-9e3f30df6a8e@wu.ac.at> <467c694d-e2ba-3f52-e1e6-41512c34811a@oracle.com> Message-ID: <9e385523-589e-917f-44e3-93ef887c67be@wu.ac.at> On 24.01.2018 20:48, Alan Bateman wrote: > On 24/01/2018 15:42, Rony G. Flatscher wrote: >> >> OK, now add to this the following class that uses p.C2 objects to access e.g. m() via it: >> >> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java >> >> public class UseC2 >> { >> ??? public static void main (String args[]) { >> ??????? p.C2 o=new p.C2(); >> ??????? System.out.println("o="+o); >> ??????? System.out.println("o.m()="+o.m()); >> ??? } >> } >> >> Compiling all three classes works. >> >> Running "UseC2" works and outputs: >> >> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2 >> o=p.C2 at 66048bfd >> o.m()=-1 >> >> So it is possible to access m() via the p.C2 object from UseC2. > That's right and this is the reason for moving to the simpler example. --- > > To be absolutely sure then you should decompile C2.class to make sure that there isn't a bridge > method calling C1's m2(). If you change m() to be final then that will keep the bridge method from > complicating the picture. > > If you change UseC2 to use core reflection and you hit the issue because the Method object you get > is p.C1.m(). Attempting to invoke this will fail with IllegalAccessException. In your other mail > you show a code fragment where it catches exceptions and calls setAccessible - I'll guess that > this may have been masking the issue in the Rexx bridge. The logic to find a matching Method object is to start out in the object's class for which reflective invocation is needed. Then using getDeclaredMethods() and look for the public method (with the sought name and parameters types that can be used for the supplied arguments), if not found, going up its superclass and repeating the process until a matching Method object is found. Being sure that the found public Method object is the appropriate one setAccess() makes it accessible. > For completeness then you may want to try out the new reflection API. I realize you have to > compile to JDK 6 but I think you'll find it will work the same way as the invokevirtual that o.m() > compiles to. It will not be a problem to supply different reflection versions after the rewrite. > ??? MethodHandles.Lookup lookup = MethodHandles.lookup(); > ??? MethodType mt = MethodType.methodType(int.class); > ??? MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt); > ??? Object res = mh.invoke(o); Thank you very much for this snippet, it really helped me a lot to explore MethodHandles in the context of my problem! After some testing, it looks like it will allow me to solve the problem, although for a price consisting of learning a totally different way? (which is o.k. for me personally) of achieving what used to be simple and safe (accessing only public members via reflection), but potentially also incurring a performance penalty compared to using only reflection. The expected performance penalty would be due to the fact that reflection cannot be by passed: the Rexx bridge supplies the member's names usually in uppercase and primitive values as plain strings, such that one must use reflection to locate candidate members (need to be public and caselessly carrying the Rexx supplied member name), then check whether the Rexx supplied arguments can be coerced to the parameter types of the candidate, and if so access it; if not, check the next candidate. Now, further steps will be necessary in addition: among them create the MethodType, get the MethodHandle, render the arguments and invoke it. --- Is there a possibility to have invoke arguments to MethodHandles be relayed, say to some method RexxValue2JavaValue, such that one can receive the MethodType, the Rexx arguments and return the matching type-converted values that should then be used for the invoke operation? ---rony From mandy.chung at oracle.com Wed Jan 31 00:27:31 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 30 Jan 2018 16:27:31 -0800 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified Message-ID: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ Improve the error message when invalid argument is specified to --suggest-providers option. $ jlink --add-modules java.base --suggest-providers java.security.Provider --output myimage Error: --suggest-providers does not accept "--output myimage" argument Mandy From mandy.chung at oracle.com Wed Jan 31 00:31:25 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 30 Jan 2018 16:31:25 -0800 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified In-Reply-To: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> References: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> Message-ID: <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> (resend with fixed formatting) Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ Improve the error message when invalid argument is specified to --suggest-providers option. $ jlink --add-modules java.base --suggest-providers java.security.Provider --output myimage Error: --suggest-providers does not accept "--output myimage" argument Mandy From lance.andersen at oracle.com Wed Jan 31 01:09:38 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 30 Jan 2018 20:09:38 -0500 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified In-Reply-To: <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> References: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> Message-ID: Looks OK Mandy could also tweak the message to ?does not accept the ?-output?? or ?does not support the ?-output?? Best Lance > On Jan 30, 2018, at 7:31 PM, mandy chung wrote: > > (resend with fixed formatting) > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ > > > Improve the error message when invalid argument is specified to > --suggest-providers option. > > > $ jlink --add-modules java.base --suggest-providers java.security.Provider --output myimage > Error: --suggest-providers does not accept "--output myimage" argument > > > Mandy > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From sundararajan.athijegannathan at oracle.com Wed Jan 31 04:56:17 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 31 Jan 2018 10:26:17 +0530 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified In-Reply-To: <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> References: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> Message-ID: <5A714C71.4020105@oracle.com> +1 -Sundar On 31/01/18, 6:01 AM, mandy chung wrote: > (resend with fixed formatting) > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ > > > Improve the error message when invalid argument is specified to > --suggest-providers option. > > > $ jlink --add-modules java.base --suggest-providers > java.security.Provider --output myimage > Error: --suggest-providers does not accept "--output myimage" argument > > > Mandy > From Alan.Bateman at oracle.com Wed Jan 31 07:36:53 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 31 Jan 2018 07:36:53 +0000 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified In-Reply-To: <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> References: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> Message-ID: <7a75727b-1642-e87e-8b85-613b9c997bcc@oracle.com> On 31/01/2018 00:31, mandy chung wrote: > (resend with fixed formatting) > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ > > > Improve the error message when invalid argument is specified to > --suggest-providers option. > > > $ jlink --add-modules java.base --suggest-providers > java.security.Provider --output myimage > Error: --suggest-providers does not accept "--output myimage" argument Looks good. From mandy.chung at oracle.com Wed Jan 31 18:17:08 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 31 Jan 2018 10:17:08 -0800 Subject: [11] RFR JDK-8196310: jlink --suggest-providers fails with missing resource if --output specified In-Reply-To: References: <0f43203a-5ec2-03af-3018-278adbd72035@oracle.com> <1f157cc8-09a9-1c19-0eca-26ed584bf9d2@oracle.com> Message-ID: <1c75c0e9-faa9-02d3-fd60-8f6e997fd0fa@oracle.com> Hi Lance, --suggest-providers takes an optional argument (comma-separated list of service types) and anything following it is not accepted. So "does not accept" is appropriate in this error message. Mandy On 1/30/18 5:09 PM, Lance Andersen wrote: > Looks OK Mandy > > could also tweak the message to > > ?does not accept *the* ?-output?? > > or > > ?does not *support* *the*??-output?? > > Best > Lance >> On Jan 30, 2018, at 7:31 PM, mandy chung > > wrote: >> >> (resend with fixed formatting) >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8196310/webrev.00/ >> >> >> >> Improve the error message when invalid argument is specified to >> --suggest-providers option. >> >> >> $ jlink --add-modules java.base --suggest-providers >> java.security.Provider --output myimage >> Error: --suggest-providers does not accept "--output myimage" argument >> >> >> Mandy >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > >