From Alan.Bateman at oracle.com Mon Aug 1 08:53:45 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 Aug 2022 09:53:45 +0100 Subject: Could we have --exclude-modules option? In-Reply-To: <2024878946.190573.1658409893388@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> Message-ID: <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> On 21/07/2022 14:24, Andrejus Chaliapinas wrote: > Hi, > > While dealing with long list of Jar dependencies for complex Maven > project and trying to resolve some of Java Modules conflicts - I'm > finding that sometimes it could be useful to exclude some modules > without yet removing actual jar file from aggregated directory of > dependencies. > > In that regards something like --exclude-modules would help initially > to evaluate impact and later allow adjust build/dependencies > resolution logic. What do you think? > I don't think this make sense as code will not compile or run if you dependences are removed. Have you found modules where the author has included `requires` clauses for modules that aren't actually required, or maybe you are dealing with a module path with many automatic modules and your module is being compiled with --add-modules ALL-MODULE-PATH? One option to be aware of is the --limit-modules option [1] but I suspect it won't be useful to you right now. -Alan [1] https://openjdk.org/jeps/261#Limiting-the-observable-modules -------------- next part -------------- An HTML attachment was scrubbed... URL: From craigraw at gmail.com Fri Aug 5 10:18:30 2022 From: craigraw at gmail.com (Craig Raw) Date: Fri, 5 Aug 2022 12:18:30 +0200 Subject: JLink image libraries are not consistently reproducible Message-ID: I am having an issue concerning the reproducibility of the JVM libraries produced by JLink. When running the same build script across different machines with the same JDK, sometimes the binaries and libs in the lib/runtime/bin and lib/runtime/lib folders are different. If one of the *.so files is different, they all are. However, the lib/runtime/lib/modules file produced by JLink is consistently identical. The machines running the builds are all Linux running flavours of Ubuntu. On examining the issue more closely, I notice that sometimes the files are identical to the ones in the JDK installation, and sometimes they are not. They are not significantly different, for example: JDK: -rw-rw-r-- 1 xxx xxx 16560 Aug 4 18:08 libjsig.so JLink: -rw-rw-r-- 1 xxx xxx 17136 Aug 4 19:23 libjsig.so Further, I've tried running the java binaries in each, and they differ in that one is 'sharing' and the other not: xxx at yyy:~$ ~/.sdkman/candidates/java/18.0.1-tem/bin/java --version openjdk 18.0.1 2022-04-19 OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed mode, sharing) xxx at yyy:~$ ~/project/build/image/bin/java --version openjdk 18.0.1 2022-04-19 OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed mode) What would cause JLink to modify the JVM itself? Is there a way I can achieve a consistently reproducible JVM image? The build in this case is using the org.beryx.jlink plugin with Gradle 7.5. Thanks, Craig -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Fri Aug 5 10:55:37 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Aug 2022 11:55:37 +0100 Subject: JLink image libraries are not consistently reproducible In-Reply-To: References: Message-ID: On 05/08/2022 11:18, Craig Raw wrote: > I am having an issue concerning the reproducibility of the JVM > libraries produced by JLink. When running the same build script across > different machines with the same JDK, sometimes the binaries and libs > in the lib/runtime/bin and lib/runtime/lib folders are different. If > one of the *.so files is different, they all are. However, the > lib/runtime/lib/modules file produced by JLink is consistently > identical. The machines running the builds are all Linux running > flavours of Ubuntu. > > On examining?the issue more closely, I notice that sometimes the files > are identical to the ones in the JDK installation, and sometimes they > are not. They are not significantly?different, for example: > JDK: > -rw-rw-r-- 1 xxx xxx? 16560 Aug ?4 18:08 libjsig.so > > JLink: > -rw-rw-r-- 1 xxx xxx ?17136 Aug ?4 19:23 libjsig.so > > Further, I've tried running the java binaries in each, and they differ > in that one is 'sharing' and the other not: > > xxx at yyy:~$ ~/.sdkman/candidates/java/18.0.1-tem/bin/java --version > openjdk 18.0.1 2022-04-19 > OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) > OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed > mode, sharing) > > xxx at yyy:~$ ~/project/build/image/bin/java --version > openjdk 18.0.1 2022-04-19 > OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) > OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed mode) > > What would cause JLink to modify the JVM itself? Is there a way I can > achieve a consistently reproducible JVM image? > I assume the jlink command used to generate project/build/image doesn't specify --generate-cds-archive so that will explain why you don't see "sharing" in the java -version output. You can use project/build/image/bin/java -Xshare:dump to do the generate it too. For the .so file it is possible that your jlink command specifies -strip-debug or --strip-native-debug-symbols? That might explain why the sizes are different to the initial image. The stripping of debug symbols from shared objects runs the objcopy command. Off-hand I don't know if this is reproducible but maybe there are different versions on the Ubuntu systems? If you aren't stripping debug symbols then another possibility is that that the package modules (jmod files) used on the different versions are identical - this is something you can quickly check. -Alan. From craigraw at gmail.com Mon Aug 8 09:43:00 2022 From: craigraw at gmail.com (Craig Raw) Date: Mon, 8 Aug 2022 11:43:00 +0200 Subject: JLink image libraries are not consistently reproducible In-Reply-To: References: Message-ID: Thanks for the quick and useful response! It was indeed the stripping of native debug symbols that was the cause (due to different versions of objcopy on the build machines). I would suggest those creating reproducible builds use only --strip-java-debug-attributes, which (as I understand it) should be platform-independent. Craig On Fri, Aug 5, 2022 at 12:55 PM Alan Bateman wrote: > On 05/08/2022 11:18, Craig Raw wrote: > > I am having an issue concerning the reproducibility of the JVM > > libraries produced by JLink. When running the same build script across > > different machines with the same JDK, sometimes the binaries and libs > > in the lib/runtime/bin and lib/runtime/lib folders are different. If > > one of the *.so files is different, they all are. However, the > > lib/runtime/lib/modules file produced by JLink is consistently > > identical. The machines running the builds are all Linux running > > flavours of Ubuntu. > > > > On examining the issue more closely, I notice that sometimes the files > > are identical to the ones in the JDK installation, and sometimes they > > are not. They are not significantly different, for example: > > JDK: > > -rw-rw-r-- 1 xxx xxx 16560 Aug 4 18:08 libjsig.so > > > > JLink: > > -rw-rw-r-- 1 xxx xxx 17136 Aug 4 19:23 libjsig.so > > > > Further, I've tried running the java binaries in each, and they differ > > in that one is 'sharing' and the other not: > > > > xxx at yyy:~$ ~/.sdkman/candidates/java/18.0.1-tem/bin/java --version > > openjdk 18.0.1 2022-04-19 > > OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) > > OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed > > mode, sharing) > > > > xxx at yyy:~$ ~/project/build/image/bin/java --version > > openjdk 18.0.1 2022-04-19 > > OpenJDK Runtime Environment Temurin-18.0.1+10 (build 18.0.1+10) > > OpenJDK 64-Bit Server VM Temurin-18.0.1+10 (build 18.0.1+10, mixed mode) > > > > What would cause JLink to modify the JVM itself? Is there a way I can > > achieve a consistently reproducible JVM image? > > > > I assume the jlink command used to generate project/build/image doesn't > specify --generate-cds-archive so that will explain why you don't see > "sharing" in the java -version output. You can use > project/build/image/bin/java -Xshare:dump to do the generate it too. > > For the .so file it is possible that your jlink command specifies > -strip-debug or --strip-native-debug-symbols? That might explain why the > sizes are different to the initial image. The stripping of debug symbols > from shared objects runs the objcopy command. Off-hand I don't know if > this is reproducible but maybe there are different versions on the > Ubuntu systems? If you aren't stripping debug symbols then another > possibility is that that the package modules (jmod files) used on the > different versions are identical - this is something you can quickly check. > > -Alan. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrejusc at yahoo.com Mon Aug 8 13:41:06 2022 From: andrejusc at yahoo.com (Andrejus Chaliapinas) Date: Mon, 8 Aug 2022 13:41:06 +0000 (UTC) Subject: Could we have --exclude-modules option? In-Reply-To: <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> Message-ID: <1223874385.1219538.1659966066435@mail.yahoo.com> Let me elaborate a bit - recent use case was when 2 jars were exporting same package (such was fine in old classpath use case, but not anymore for Java modules case). And challenge was to figure out which exactly jar needs to be excluded from Java modules resolution graph to make overall Java Modules validation work and not lead to a conflict. Presence of either of those jars would satisfy obviously compilation, so issue here is not about compilation itself. With proposed exclude approach it would be much easier to exclude jar1 - do various evaluations/tests and if anything failing - switch to jar2. And do all that yet not touching Maven (other build tool) dependencies aggregation inside some folder/local repo. And once reaching point of passing set - adjust later dependencies in other places. And in that regards mentioned --limit-modules would be longer path to achieve above fast evaluation of impact. On Monday, August 1, 2022 at 09:53:53 AM GMT+1, Alan Bateman wrote: On 21/07/2022 14:24, Andrejus Chaliapinas wrote: Hi, While dealing with long list of Jar dependencies for complex Maven project and trying to resolve some of Java Modules conflicts - I'm finding that sometimes it could be useful to exclude some modules without yet removing actual jar file from aggregated directory of dependencies. In that regards something like --exclude-modules would help initially to evaluate impact and later allow adjust build/dependencies resolution logic. What do you think? I don't think this make sense as code will not compile or run if you dependences are removed. Have you found modules where the author has included `requires` clauses for modules that aren't actually required, or maybe you are dealing with a module path with many automatic modules and your module is being compiled with --add-modules ALL-MODULE-PATH? One option to be aware of is the --limit-modules option [1] but I suspect it won't be useful to you right now. -Alan [1] https://openjdk.org/jeps/261#Limiting-the-observable-modules -------------- next part -------------- An HTML attachment was scrubbed... URL: From kr.ashish8196 at gmail.com Mon Aug 8 21:17:59 2022 From: kr.ashish8196 at gmail.com (Kumar Ashish) Date: Tue, 9 Aug 2022 02:47:59 +0530 Subject: [JEP 403] Use case for a new standard replacement Message-ID: Hi, I have been using https://github.com/uber/cadence-java-client in my service, and it contains two illegal access operations. I am trying to upgrade to java 17 in my service, but am unable to do so because of these operations. These operations have been mentioned in this issue, https://github.com/uber/cadence-java-client/issues/647 . I am not sure if this has already been queried but what should I do to solve this? Regards, Kumar Ashish -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Mon Aug 8 21:46:48 2022 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 8 Aug 2022 23:46:48 +0200 (CEST) Subject: [JEP 403] Use case for a new standard replacement In-Reply-To: References: Message-ID: <965284922.19939754.1659995208219.JavaMail.zimbra@u-pem.fr> > From: "Kumar Ashish" > To: jigsaw-dev at openjdk.org > Sent: Monday, August 8, 2022 11:17:59 PM > Subject: [JEP 403] Use case for a new standard replacement > Hi, > I have been using [ https://github.com/uber/cadence-java-client | > https://github.com/uber/cadence-java-client ] in my service, and it contains > two illegal access operations. I am trying to upgrade to java 17 in my service, > but am unable to do so because of these operations. These operations have been > mentioned in this issue, [ > https://github.com/uber/cadence-java-client/issues/647 | > https://github.com/uber/cadence-java-client/issues/647 ] . I am not sure if > this has already been queried but what should I do to solve this? Hi, if you control the command line, you can use --add-opens, something like --add-opens java.base/java.lang=ALL-UNNAMED > Regards, > Kumar Ashish R?mi -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Mon Aug 8 23:06:52 2022 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 9 Aug 2022 01:06:52 +0200 (CEST) Subject: Could we have --exclude-modules option? In-Reply-To: <1223874385.1219538.1659966066435@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> Message-ID: <1753841265.19956276.1660000012222.JavaMail.zimbra@u-pem.fr> > From: "Andrejus Chaliapinas" > To: "jigsaw-dev" > Sent: Monday, August 8, 2022 3:41:06 PM > Subject: Re: Could we have --exclude-modules option? > Let me elaborate a bit - recent use case was when 2 jars were exporting same > package (such was fine in old classpath use case, but not anymore for Java > modules case). And challenge was to figure out which exactly jar needs to be > excluded from Java modules resolution graph to make overall Java Modules > validation work and not lead to a conflict. > Presence of either of those jars would satisfy obviously compilation, so issue > here is not about compilation itself. With proposed exclude approach it would > be much easier to exclude jar1 - do various evaluations/tests and if anything > failing - switch to jar2. And do all that yet not touching Maven (other build > tool) dependencies aggregation inside some folder/local repo. And once reaching > point of passing set - adjust later dependencies in other places. > And in that regards mentioned --limit-modules would be longer path to achieve > above fast evaluation of impact. It's perhaps easier to just merge the two jars into a unique one, this will resolve any split jar conflicts. R?mi > On Monday, August 1, 2022 at 09:53:53 AM GMT+1, Alan Bateman > wrote: > On 21/07/2022 14:24, Andrejus Chaliapinas wrote: >> Hi, >> While dealing with long list of Jar dependencies for complex Maven project and >> trying to resolve some of Java Modules conflicts - I'm finding that sometimes >> it could be useful to exclude some modules without yet removing actual jar file >> from aggregated directory of dependencies. >> In that regards something like --exclude-modules would help initially to >> evaluate impact and later allow adjust build/dependencies resolution logic. >> What do you think? > I don't think this make sense as code will not compile or run if you dependences > are removed. Have you found modules where the author has included `requires` > clauses for modules that aren't actually required, or maybe you are dealing > with a module path with many automatic modules and your module is being > compiled with --add-modules ALL-MODULE-PATH? > One option to be aware of is the --limit-modules option [1] but I suspect it > won't be useful to you right now. > -Alan > [1] [ https://openjdk.org/jeps/261#Limiting-the-observable-modules | > https://openjdk.org/jeps/261#Limiting-the-observable-modules ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Tue Aug 9 06:35:23 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 Aug 2022 07:35:23 +0100 Subject: [JEP 403] Use case for a new standard replacement In-Reply-To: References: Message-ID: On 08/08/2022 22:17, Kumar Ashish wrote: > Hi, > I have been using https://github.com/uber/cadence-java-client in my > service, and it contains two illegal access operations. I am trying to > upgrade to java 17 in my service, but am unable to do so because of > these operations. These operations have been mentioned in this issue, > https://github.com/uber/cadence-java-client/issues/647 . I am not sure > if this has already been queried but what should I do to solve this? In this case, it looks like the maintainer of this project will need to re-examine how their serialize Throwables. It's really bizarre that serializing a Throwable to JSON format would first hack the Throwable's cause. So I think it would be better to fix the issue there rather than trying to workaround it with --add-opens options. -Alan From Alan.Bateman at oracle.com Tue Aug 9 11:36:52 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 Aug 2022 12:36:52 +0100 Subject: Could we have --exclude-modules option? In-Reply-To: <1223874385.1219538.1659966066435@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> Message-ID: <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> On 08/08/2022 14:41, Andrejus Chaliapinas wrote: > Let me elaborate a bit - recent use case was when 2 jars were > exporting same package (such was fine in old classpath use case, but > not anymore for Java modules case). And challenge was to figure out > which exactly jar needs to be excluded from Java modules resolution > graph to make overall Java Modules validation work and not lead to a > conflict. > > Presence of either of those jars would satisfy obviously compilation, > so issue here is not about compilation itself. With proposed exclude > approach it would be much easier to exclude jar1 - do various > evaluations/tests and if anything failing - switch to jar2. And do all > that yet not touching Maven (other build tool) dependencies > aggregation inside some folder/local repo. And once reaching point of > passing set - adjust later dependencies in other places. > Have you tried java --validate-modules? This option will scan the module path split package for other issues so that you get a dump of all issues. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrejusc at yahoo.com Tue Aug 9 20:07:32 2022 From: andrejusc at yahoo.com (Andrejus Chaliapinas) Date: Tue, 9 Aug 2022 20:07:32 +0000 (UTC) Subject: Could we have --exclude-modules option? In-Reply-To: <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> Message-ID: <1186619735.1647646.1660075652314@mail.yahoo.com> > Have you tried java --validate-modules? This option will scan the module path split package for other issues so that you get a dump of all issues. > > -Alan Yes, that was my starting point, i.e usage of --validate-modules. But seeing outcome of it for complex dependencies set led to this my proposal actually. Cause validate shows conflicts, but elimination of them takes effort via build tooling configs. And much easier would be initially to evaluate runtime outcome via flexible exclusion capability (i.e. in a way we have for Maven dependencies exclusions for example). Andrejus -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Tue Aug 9 21:03:24 2022 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 9 Aug 2022 14:03:24 -0700 Subject: Could we have --exclude-modules option? In-Reply-To: <1186619735.1647646.1660075652314@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> <1186619735.1647646.1660075652314@mail.yahoo.com> Message-ID: <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> On 8/9/2022 1:07 PM, Andrejus Chaliapinas wrote: > > Have you tried java --validate-modules? This option will scan the > module path split package for other issues so that you get a dump of all > issues. > > Yes, that was my starting point, i.e usage of --validate-modules. But > seeing outcome of it for complex dependencies set led to this my > proposal actually. Cause validate shows conflicts, but elimination of > them takes effort via build tooling configs. And much easier would be > initially to evaluate runtime outcome via flexible exclusion capability > (i.e. in a way we have for Maven dependencies exclusions for example). Are these two JARs built from code you control? If not, then trying to figure out if your application "works" when only JAR1 is present, or only JAR2 is present, sounds difficult. And introducing --exclude-modules to "fight" the dependencies resolved by Maven is a bad idea. Alex From andrejusc at yahoo.com Tue Aug 9 21:23:09 2022 From: andrejusc at yahoo.com (Andrejus Chaliapinas) Date: Tue, 9 Aug 2022 21:23:09 +0000 (UTC) Subject: Could we have --exclude-modules option? In-Reply-To: <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> <1186619735.1647646.1660075652314@mail.yahoo.com> <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> Message-ID: <1668613787.1665166.1660080189808@mail.yahoo.com> > Are these two JARs built from code you control? No, they are all Open Source > If not, then trying to > figure out if your application "works" when only JAR1 is present, or > only JAR2 is present, sounds difficult.? It's actually not difficult - set of tests runs over and checks runtime behaviour. And passing extra argument to either Surefire for UTs or Failsafe for ITs/etc. plugins configs is trivial. > And introducing > --exclude-modules to "fight" the dependencies resolved by Maven is a bad > idea. I see this option as extra flexibility for complex projects. It's not about "fight" here, but about fast way of figuring out what is actually needed/suitable and allowing gradual migrations. Cause dealing with Maven exclusions in POMs (even if that is final goal to fully eliminate conflicts) is more difficult till some extent. One side use case here is presence of jars with key word in their names (like "byte" or "native"), which are invalid from Java modules angle, but quite often are yet Automatic as not migrated to support Java modules. So then workaround is either injecting extra Manifest entry by some Maven plugin or renaming overall jar. Both activities are less trivial comparing to proposed option. Andrejus On Tuesday, August 9, 2022, 10:03:56 PM GMT+1, Alex Buckley wrote: On 8/9/2022 1:07 PM, Andrejus Chaliapinas wrote: > > Have you tried java --validate-modules? This option will scan the > module path split package for other issues so that you get a dump of all > issues. > > Yes, that was my starting point, i.e usage of --validate-modules. But > seeing outcome of it for complex dependencies set led to this my > proposal actually. Cause validate shows conflicts, but elimination of > them takes effort via build tooling configs. And much easier would be > initially to evaluate runtime outcome via flexible exclusion capability > (i.e. in a way we have for Maven dependencies exclusions for example). Are these two JARs built from code you control? If not, then trying to figure out if your application "works" when only JAR1 is present, or only JAR2 is present, sounds difficult. And introducing --exclude-modules to "fight" the dependencies resolved by Maven is a bad idea. Alex From alex.buckley at oracle.com Tue Aug 9 23:35:03 2022 From: alex.buckley at oracle.com (Alex Buckley) Date: Tue, 9 Aug 2022 16:35:03 -0700 Subject: [External] : Re: Could we have --exclude-modules option? In-Reply-To: <1668613787.1665166.1660080189808@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> <1186619735.1647646.1660075652314@mail.yahoo.com> <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> <1668613787.1665166.1660080189808@mail.yahoo.com> Message-ID: <7f3c5b26-a744-050e-6b2f-282cfbf87bac@oracle.com> On 8/9/2022 2:23 PM, Andrejus Chaliapinas wrote: >> Are these two JARs built from code you control? > > No, they are all Open Source Are they third-party dependencies of your application, or fourth-party? Also, what did their maintainer say when you pointed out the existence of split packages, and how they make it difficult to use the JARs as automatic modules? When you're faced with code that's aggressively unmodular, it seems attractive for the JDK to come swinging to the rescue with a power-user command-line option to tweak the environment just so. But these options accumulate in scripts (and then get forgotten about), and they interact with other options in surprising ways that generate new requests for further tweaks. Options that duplicate functionality found outside the JDK (e.g., --exclude-modules vs. POM ) impose an even bigger cost, as people need to understand the differences, get confused when the option doesn't work like the external tool, etc. The best and most maintainable path is to fix the legacy JARs so they work cleanly as automatic modules. Alex >> If not, then trying to >> figure out if your application "works" when only JAR1 is present, or >> only JAR2 is present, sounds difficult. > > It's actually not difficult - set of tests runs over and checks runtime behaviour. And passing extra argument to either Surefire for UTs or Failsafe for ITs/etc. plugins configs is trivial. > >> And introducing >> --exclude-modules to "fight" the dependencies resolved by Maven is a bad >> idea. > > I see this option as extra flexibility for complex projects. It's not about "fight" here, but about fast way of figuring out what is actually needed/suitable and allowing gradual migrations. Cause dealing with Maven exclusions in POMs (even if that is final goal to fully eliminate conflicts) is more difficult till some extent. > > One side use case here is presence of jars with key word in their names (like "byte" or "native"), which are invalid from Java modules angle, but quite often are yet Automatic as not migrated to support Java modules. So then workaround is either injecting extra Manifest entry by some Maven plugin or renaming overall jar. Both activities are less trivial comparing to proposed option. > > Andrejus > > > > On Tuesday, August 9, 2022, 10:03:56 PM GMT+1, Alex Buckley wrote: > > > > > > On 8/9/2022 1:07 PM, Andrejus Chaliapinas wrote: >>> Have you tried java --validate-modules? This option will scan the >> module path split package for other issues so that you get a dump of all >> issues. >> >> Yes, that was my starting point, i.e usage of --validate-modules. But >> seeing outcome of it for complex dependencies set led to this my >> proposal actually. Cause validate shows conflicts, but elimination of >> them takes effort via build tooling configs. And much easier would be >> initially to evaluate runtime outcome via flexible exclusion capability >> (i.e. in a way we have for Maven dependencies exclusions for example). > > Are these two JARs built from code you control? If not, then trying to > figure out if your application "works" when only JAR1 is present, or > only JAR2 is present, sounds difficult. And introducing > --exclude-modules to "fight" the dependencies resolved by Maven is a bad > idea. > > Alex From andrejusc at yahoo.com Wed Aug 10 19:04:54 2022 From: andrejusc at yahoo.com (Andrejus Chaliapinas) Date: Wed, 10 Aug 2022 19:04:54 +0000 (UTC) Subject: [External] : Re: Could we have --exclude-modules option? In-Reply-To: <7f3c5b26-a744-050e-6b2f-282cfbf87bac@oracle.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> <1186619735.1647646.1660075652314@mail.yahoo.com> <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> <1668613787.1665166.1660080189808@mail.yahoo.com> <7f3c5b26-a744-050e-6b2f-282cfbf87bac@oracle.com> Message-ID: <555799958.10360.1660158294941@mail.yahoo.com> > Are they third-party dependencies of your application, or fourth-party? It's a mix of both > Also, what did their maintainer say when you pointed out the existence > of split packages, and how they make it difficult to use the JARs as > automatic modules? Well, I'm not sure everyone is buying importance of Java modules usage with extra headache involved. So I expect some of those to continue be just old Java8 jars unless some Java 18+ will deprecate classpath usage completely, which I doubt. > When you're faced with code that's aggressively unmodular, it seems > attractive for the JDK to come swinging to the rescue with a power-user > command-line option to tweak the environment just so. But these options > accumulate in scripts (and then get forgotten about)... In this case it's not a tweak, but similar functional parity with Maven where we could do flexible exclusions. There were lot of approaches of Classloaders filtering in Java8 cases and I don't see why we cannot have similar here. Andrejus From alex.buckley at oracle.com Wed Aug 10 21:36:20 2022 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 10 Aug 2022 14:36:20 -0700 Subject: [External] : Re: Could we have --exclude-modules option? In-Reply-To: <555799958.10360.1660158294941@mail.yahoo.com> References: <2024878946.190573.1658409893388.ref@mail.yahoo.com> <2024878946.190573.1658409893388@mail.yahoo.com> <1bc1997f-4eba-dee7-b11a-6742fdd836f3@oracle.com> <1223874385.1219538.1659966066435@mail.yahoo.com> <402d2947-ab94-11c4-b296-34288f9f5c47@oracle.com> <1186619735.1647646.1660075652314@mail.yahoo.com> <3723fb78-3360-81a6-c35a-6b72a0064977@oracle.com> <1668613787.1665166.1660080189808@mail.yahoo.com> <7f3c5b26-a744-050e-6b2f-282cfbf87bac@oracle.com> <555799958.10360.1660158294941@mail.yahoo.com> Message-ID: On 8/10/2022 12:04 PM, Andrejus Chaliapinas wrote: >> Also, what did their maintainer say when you pointed out the >> existence of split packages, and how they make it difficult to use >> the JARs as automatic modules? > > Well, I'm not sure everyone is buying importance of Java modules > usage with extra headache involved. So I expect some of those to > continue be just old Java8 jars unless some Java 18+ will deprecate > classpath usage completely, which I doubt. An impressive majority of old Java 8 JARs work as automatic modules on Java 9+ with no effort from the maintainer. However, it was never the aim to make 100% of them work. JARs that make use of poor practices -- whether split packages, access to JDK internals, tight dependence on version strings, etc -- will find it hard to run on new versions of Java. We're not going to keep adding mechanisms, such as command-line options or additional manifest entries, to catch the final 5%, 1%, etc -- especially when tools that set up the modulepath can already mitigate some of the JARs' problems. Alex From rdebusscher at gmail.com Thu Aug 25 18:24:46 2022 From: rdebusscher at gmail.com (Rudy De Busscher) Date: Thu, 25 Aug 2022 20:24:46 +0200 Subject: Generic MethodHandle.lookup utility method and modules Message-ID: Hi All, I'm playing around with MethodHandle.lookup and LambdaMetafactory to have some generic utility method to access fields (through the getters and setters) of some arbitrary class. (as alternative for reflection as this should be faster) Code is working fine in classpath mode, but when using Java modules, I get errors saying 'access to public member failed' although module info exports the class. I believe this is by design because I read the following sentence in the javadoc of MethodHandles.Lookup "*Teleporting to some third module drops all accesses. *" I seek confirmation that it is indeed not possible what I'm trying (JDK 11/JDK 17) *Setup* Module R, Class with calls to MethodHandle.lookup and LambdaMetafactory Module M, contains simple pojos, public setters and getters for each property. module info contains exports and open clauses for the pojo package(s). Module App contains the main method, calling the utility class in Module R with Pojos of Module M. *Code* Since I have *MethodHandles.Lookup lookup = MethodHandles.publicLookup().in(beanClass);* Within a class of Module R, called from Module App, where beanClass is a class within M I assume I get this error due to the 'teleporting to third module' access to public member failed: be.atbash.poc.reflection.Pojo.getData()String/invokeVirtual, from be.atbash.json.accessor.FastPropertyMemberAccessor (module reflection) When I use MethodHandles.Lookup lookup = MethodHandles.lookup().in(beanClass); I have as error symbolic reference class is not accessible: class be.atbash.poc.reflection.Pojo, from be.atbash.poc.reflection.Pojo/noaccess (module model) which also indicate the access check is an issue ( xxx/noaccess) *Question* Thanks for any additional insight I might miss or confirmation it is not possible what I try when using Modules. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Thu Aug 25 23:25:00 2022 From: mandy.chung at oracle.com (mandy.chung at oracle.com) Date: Thu, 25 Aug 2022 16:25:00 -0700 Subject: Generic MethodHandle.lookup utility method and modules In-Reply-To: References: Message-ID: <88112cd2-b646-dd5c-1579-a27b5a2905b0@oracle.com> When you teleport the lookup from module R class C to another module M class D, the cross-module access check ensures that the resulting Lookup has access to public types that both R/C and M/D can access (i.e. their intersection). The table in the cross-module access checks section at: https://download.java.net/java/early_access/jdk20/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#module-access-check describes the allowed modes of the resulting Lookup after teleporting. On 8/25/22 11:24 AM, Rudy De Busscher wrote: > Hi All, > > I'm playing around with MethodHandle.lookup and LambdaMetafactory to > have some generic utility method to access fields (through the getters > and setters) of some arbitrary class. (as alternative for reflection > as this should be faster) > > Code is working fine in classpath mode, but when using Java modules, I > get errors?saying 'access to public member failed' although module > info exports the class. > > I believe this is by design because I read the following sentence in > the javadoc of MethodHandles.Lookup "*Teleporting to some third module > drops all accesses. *" > Yes.?? Teleporting across modules can only decrease access but cannot increase it. > I seek confirmation that it is indeed not possible what I'm trying > (JDK 11/JDK 17) > > _Setup_ > > Module R, Class with calls to MethodHandle.lookup and LambdaMetafactory > > Module M, contains simple pojos, public setters and getters for each > property. module info contains exports and open clauses for the pojo > package(s). > > Module App contains the main method, calling the utility class in > Module R with Pojos of Module M. > > _Code_ > > Since I have > > /MethodHandles.Lookup lookup = > MethodHandles.publicLookup().in(beanClass);/ > A public Lookup has UNCONDITIONAL mode that can access public type in all modules when the type is in a package that is exported unconditionally. Teleporting to a different module does not change its allowed mode. > Within a class of Module R, called from Module App, where beanClass is > a class within M I assume I get this error due to the 'teleporting to > third module' > > access to public member failed: > be.atbash.poc.reflection.Pojo.getData()String/invokeVirtual, from > be.atbash.json.accessor.FastPropertyMemberAccessor (module reflection) > > Is be.atbash.poc.reflection.Pojo exported unconditionally? > When I use > > > MethodHandles.Lookup lookup = MethodHandles.lookup().in(beanClass); > > I have as error > > > symbolic reference class is not accessible: class > be.atbash.poc.reflection.Pojo, from > be.atbash.poc.reflection.Pojo/noaccess (module model) > > > which also indicate the access check is an issue ( xxx/noaccess) > Is this error message from JDK 11?? Is it a different message when running with JDK 17? Mandy > > _Question_ > > > Thanks for any additional insight?I might miss or confirmation it is > not possible what I try when using Modules. > -------------- next part -------------- An HTML attachment was scrubbed... URL: