From mandy.chung at oracle.com Fri Jul 17 18:45:27 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 17 Jul 2020 11:45:27 -0700 Subject: RFR JDK-8217527: jmod hash does not work if --hash-module does not include the target module Message-ID: Webrev: http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8217527/webrev.00/ The current implementation jmod hash ignores the target jmod file if specified. Therefore even B requires A: $ jmod hash --hash-module B --module-path jmods jmods/A.jmod This jmod hash command finds no module to record hashes. The patch fixes jmod hash such that if the target jmod file is specified, it is the module to record the hashes and it must have no outgoing read edges to the modules on the application path. If the target jmod file is not specified, jmod hash will find from the resolved modules that have no outgoing edges and match the given pattern to record hashes of the modules requiring it directly or indirectly. jmod hash command should also report no hash recorded if no module found to record hashes. Thanks Mandy From Alan.Bateman at oracle.com Sat Jul 18 08:09:56 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 18 Jul 2020 09:09:56 +0100 Subject: RFR JDK-8217527: jmod hash does not work if --hash-module does not include the target module In-Reply-To: References: Message-ID: <246fb553-15a2-d844-b68a-dd96f16431ba@oracle.com> On 17/07/2020 19:45, Mandy Chung wrote: > Webrev: > http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8217527/webrev.00/ > > The current implementation jmod hash ignores the target jmod file if > specified. > > Therefore even B requires A: > > $ jmod hash --hash-module B --module-path jmods jmods/A.jmod > > This jmod hash command finds no module to record hashes. > > The patch fixes jmod hash such that if the target jmod file is > specified, it is the module to record the hashes and it must have no > outgoing read edges to the modules on the application path. If the > target jmod file is not specified, jmod hash will find from the > resolved modules that have no outgoing edges and match the given > pattern to record hashes of the modules requiring it directly or > indirectly. > > jmod hash command should also report no hash recorded if no module > found to record hashes. This looks good good, and good to have it report when it doesn't record any hashes. -Alan From mgridinas at gmail.com Mon Jul 20 20:25:05 2020 From: mgridinas at gmail.com (Mantas Gridinas) Date: Mon, 20 Jul 2020 20:25:05 +0000 Subject: Debugging while using the module system Message-ID: Hi! Recently I've moved my project to the module system from regular classpath jars. Currently I'm using Adopt OpenJDK11 builds and for debugging I provide the -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=localhost:5005 command line argument while starting my application. As a debugging client I use intellij idea's remote debugging functionality. Sometimes (I am not sure myself what exactly causes this), while debugging my project, I get into an issue where the debugger cannot access some classes, their fields or methods. Since right now I cannot provide the exact error message, I would like to ask: what are the limitations of the debugger implementation, provided by the JDK, when used on a runtime with a module system rather than a classpath based system? Cheers. From Alan.Bateman at oracle.com Tue Jul 21 06:52:24 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 21 Jul 2020 07:52:24 +0100 Subject: Debugging while using the module system In-Reply-To: References: Message-ID: <318a3e44-aa36-bf9c-0857-819cd6a70d33@oracle.com> On 20/07/2020 21:25, Mantas Gridinas wrote: > Hi! > > Recently I've moved my project to the module system from regular > classpath jars. Currently I'm using Adopt OpenJDK11 builds and for > debugging I provide the > -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=localhost:5005 > command line argument while starting my application. As a debugging > client I use intellij idea's remote debugging functionality. Sometimes > (I am not sure myself what exactly causes this), while debugging my > project, I get into an issue where the debugger cannot access some > classes, their fields or methods. Since right now I cannot provide the > exact error message, I would like to ask: what are the limitations of > the debugger implementation, provided by the JDK, when used on a > runtime with a module system rather than a classpath based system? > Debuggers should work with modular applications. Under the covers, the JDWP agent that you specify with -agentlib uses the JVM Tool Interface and the Java Native Interface that aren't subject to access control. Maybe IDEA is spinning byte code or classes in the target VM? Have you tried submitting a bug to Jetbrains on this? -Alan From mgridinas at gmail.com Tue Jul 21 18:48:19 2020 From: mgridinas at gmail.com (Mantas Gridinas) Date: Tue, 21 Jul 2020 21:48:19 +0300 Subject: Debugging while using the module system In-Reply-To: <318a3e44-aa36-bf9c-0857-819cd6a70d33@oracle.com> References: <318a3e44-aa36-bf9c-0857-819cd6a70d33@oracle.com> Message-ID: Now that you mention it, this does occur while trying to use "Evaluate expression" which spins up bytecode and evaluates it at run time. I haven't contacted Jetbrains on this yet, but I'll open an issue on their tracker as well. Cheers. (reposted on mailing list since) On Tue, Jul 21, 2020 at 9:52 AM Alan Bateman wrote: > > On 20/07/2020 21:25, Mantas Gridinas wrote: > > Hi! > > > > Recently I've moved my project to the module system from regular > > classpath jars. Currently I'm using Adopt OpenJDK11 builds and for > > debugging I provide the > > -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=localhost:5005 > > command line argument while starting my application. As a debugging > > client I use intellij idea's remote debugging functionality. Sometimes > > (I am not sure myself what exactly causes this), while debugging my > > project, I get into an issue where the debugger cannot access some > > classes, their fields or methods. Since right now I cannot provide the > > exact error message, I would like to ask: what are the limitations of > > the debugger implementation, provided by the JDK, when used on a > > runtime with a module system rather than a classpath based system? > > > Debuggers should work with modular applications. Under the covers, the > JDWP agent that you specify with -agentlib uses the JVM Tool Interface > and the Java Native Interface that aren't subject to access control. > Maybe IDEA is spinning byte code or classes in the target VM? Have you > tried submitting a bug to Jetbrains on this? > > -Alan From Alan.Bateman at oracle.com Wed Jul 22 09:36:19 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 22 Jul 2020 10:36:19 +0100 Subject: Debugging while using the module system In-Reply-To: References: <318a3e44-aa36-bf9c-0857-819cd6a70d33@oracle.com> Message-ID: On 21/07/2020 19:48, Mantas Gridinas wrote: > Now that you mention it, this does occur while trying to use "Evaluate > expression" which spins up bytecode and evaluates it at run time. I > haven't contacted Jetbrains on this yet, but I'll open an issue on > their tracker as well. > This may be a good topic to continue on serviceability-dev as there may be additions to be JDWP and JDI to support some of these debugger use-cases. JVM TI has the module functions that are needed expand readability or the packages that are exported or opened for tools doing instrumentation and it's exactly the same support needed when generating code in the target VM to do expression evaluation. The summary is the it's not com.sun.jdi.VirtualMachine API or JDWP and it probably should. -Alan. From sormuras at gmail.com Wed Jul 29 12:49:14 2020 From: sormuras at gmail.com (Christian Stein) Date: Wed, 29 Jul 2020 14:49:14 +0200 Subject: javac --module option and stale classes Message-ID: Hi, when using the --module option of javac, the Java source files of a module are automatically added to the list of to be compiled files, checking timestamps. This works as expected as long as existing source files are changed and/or new source files are added. If a Java source file is deleted from a module its associated class files are left in place (within the -d destination directory). This leads to unexpected and wrong results in downstream tool calls - for example, a JAR file with stale/orphaned classes as described by [0]. The straightforward "solution" is to delete the -D directory prior to every call of javac --module ... which leads to a "rebuild" of all modules and effectively by-passes the gains provided by the timestamp check. This "solution" can be fine-tuned in some ways via remembering checksums and other external measures. I wonder whether the logic at [1] is a) the sole place that controls this feature and b) whether it can be enhanced to also prune stale classes from previous javac runs. Cheers, Christian [0] https://github.com/sormuras/bach/issues/158 [1] https://github.com/openjdk/jdk/blob/76baa501faa11c934f7ee8e463e13d098ef90518/src/jdk.compiler/share/classes/com/sun/tools/javac/main/Arguments.java#L426-L435 From jonathan.gibbons at oracle.com Wed Jul 29 16:39:16 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 29 Jul 2020 09:39:16 -0700 Subject: javac --module option and stale classes In-Reply-To: References: Message-ID: On 7/29/20 5:49 AM, Christian Stein wrote: > I wonder whether the logic at [1] is a) the sole place that > controls this feature and b) whether it can be enhanced > to also prune stale classes from previous javac runs. That should be the only place. The minor complication about pruning stale classes is that it is not a 1-1 mapping from .java to .class files, and that for a perfect job, you have to read the .class file to determine the likely origin of the .java file. For a good but not perfect job, you can apply heuristics to the filename, and whether it contains '$' characters or not. -- Jon From mgridinas at gmail.com Wed Jul 29 17:29:13 2020 From: mgridinas at gmail.com (Mantas Gridinas) Date: Wed, 29 Jul 2020 20:29:13 +0300 Subject: Migrating resources from classpath to modular access Message-ID: Recently I've finished watching https://www.youtube.com/watch?v=V6ycn1-QQV0 for pointers on how to explain JPMS to my collegues at work. I am aware it's a bit old, but the ideas are still present and the presentation does a good job explaining why JPMS is the way it is, what are the guarantees and why you should be using it. At around 52:40 there was a question from the audience about loading configuration files from resources on classpath, and one of the presenters responded that "libraries would provide services for configuration rather than depend on configuration files if revisited today". This peaked my interest, but sadly, the presenters didn't go deeper into what they meant by that. Did they reference the "ServiceLoader" mechanism? Cheers. From sormuras at gmail.com Thu Jul 30 06:52:31 2020 From: sormuras at gmail.com (Christian Stein) Date: Thu, 30 Jul 2020 08:52:31 +0200 Subject: javac --module option and stale classes In-Reply-To: References: Message-ID: On Wed, Jul 29, 2020 at 6:41 PM Jonathan Gibbons < jonathan.gibbons at oracle.com> wrote: > > On 7/29/20 5:49 AM, Christian Stein wrote: > > I wonder whether the logic at [1] is a) the sole place that > > controls this feature and b) whether it can be enhanced > > to also prune stale classes from previous javac runs. > > That should be the only place. > > The minor complication about pruning stale classes is that > it is not a 1-1 mapping from .java to .class files, and that for > a perfect job, you have to read the .class file to determine the > likely origin of the .java file. Are you referring to the optional "SourceFile Attribute" as described in JVMS 4.7.10? If yes, I hoped to learn about a mandatory information within the class file format that hints to its origin/compilation unit. The current implementation uses "inferBinaryName". That method returns a single value - which doesn't account for multiple types being declared in a single compilation unit: for example, a Foo.java file may (but should not) produce Bar.class and Baz.class, correct? Local experiments support this observation, as the timestamp check is not performed when the source file name and the inferred class file name(s) don't match. Meaning, Foo.java is always added to the list of "to be compiled files" as there's no Foo.class in the destination directory. So, an outline of more correct implementation could like this: a) scan each .java files for type declarations to determine all .class files being produced b) collect those .class files c) check whether .java files need to be compiled, if yes, add them to fileObjects list d) prune all .class files from the destination directory that are not collected by step b) I bet, the parsing work as described in a) will need some more CPU cycles compared to the "inferBinaryName" -- but the gain would be a more correct behaviour of javac when used with `--module` option.