From anandebeh at gmail.com Thu Jun 3 20:10:11 2021 From: anandebeh at gmail.com (Anand Beh) Date: Thu, 3 Jun 2021 16:10:11 -0400 Subject: Annotation Dependencies and Requires Static Transitive Message-ID: Hello, The cache library Caffeine recently added a full module descriptor. It has no runtime dependencies, but it depends on metadata annotations from checker-qual and errorprone, for example @NotNull and @CanIgnoreReturnValue. The module looks like this: module com.github.benmanes.caffeine { exports com.github.benmanes.caffeine.cache; exports com.github.benmanes.caffeine.cache.stats; requires static transitive com.google.errorprone.annotations; requires static transitive org.checkerframework.checker.qual; } The annotations are not required at runtime, hence static. They're visibly placed on public methods and return types, so that API clients can benefit from them for the purposes of annotation-based null analysis, kotlin interop, etc. As the annotations are part of the API, they're marked transitive. However, the "transitive" aspect imposes some requirements on users. I am wondering if there is a more correct way to declare these annotation dependencies than static transitive. One user would like to avoid the presence of these annotations at compile-time. For reference, here's the relevant discussion: https://github.com/ben-manes/caffeine/issues/535 I'm not a maintainer of caffeine, though I was involved in its modularization. Regards, Anand From forax at univ-mlv.fr Thu Jun 3 20:37:54 2021 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 3 Jun 2021 22:37:54 +0200 (CEST) Subject: Annotation Dependencies and Requires Static Transitive In-Reply-To: References: Message-ID: <1494943125.524452.1622752674213.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Anand Beh" > ?: "jigsaw-dev" > Envoy?: Jeudi 3 Juin 2021 22:10:11 > Objet: Annotation Dependencies and Requires Static Transitive > Hello, > > The cache library Caffeine recently added a full module descriptor. It > has no runtime dependencies, but it depends on metadata annotations > from checker-qual and errorprone, for example @NotNull and > @CanIgnoreReturnValue. The module looks like this: > module com.github.benmanes.caffeine { > exports com.github.benmanes.caffeine.cache; > exports com.github.benmanes.caffeine.cache.stats; > > requires static transitive com.google.errorprone.annotations; > requires static transitive org.checkerframework.checker.qual; > } > > The annotations are not required at runtime, hence static. They're > visibly placed on public methods and return types, so that API clients > can benefit from them for the purposes of annotation-based null > analysis, kotlin interop, etc. As the annotations are part of the API, > they're marked transitive. > > However, the "transitive" aspect imposes some requirements on users. I > am wondering if there is a more correct way to declare these > annotation dependencies than static transitive. > > One user would like to avoid the presence of these annotations at > compile-time. For reference, here's the relevant discussion: > https://github.com/ben-manes/caffeine/issues/535 > > I'm not a maintainer of caffeine, though I was involved in its modularization. Hi, i've used just "requires static" without "transitive" for non null/nullable annotations in the past. Thos annotations are consumed by tools, i don't think transitive is necessary. > > Regards, > Anand regards, R?mi From alex.buckley at oracle.com Thu Jun 3 21:13:00 2021 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 3 Jun 2021 14:13:00 -0700 Subject: Annotation Dependencies and Requires Static Transitive In-Reply-To: References: Message-ID: <353f78cf-6e41-9906-b3d3-ffbf76868b14@oracle.com> Even without `transitive`, requiring modules with `static` means that anyone who habitually builds their entire stack from source will still need the errorprone and checker-qual modules at compile time. There are no "run-time only" dependencies in module declarations, unless services come into play, which is not realistic here. As Remi said, `requires static` is your best bet. Annotation types that are exported by a third party such as Google, in order for people to annotate their personal codebases, are not really an API that those personal codebases need to export -- any fourth party program that wishes to inspect the annotations in your codebase needs to arrange its own dependency on the annotation types from the third party. Alex On 6/3/2021 1:10 PM, Anand Beh wrote: > Hello, > > The cache library Caffeine recently added a full module descriptor. It > has no runtime dependencies, but it depends on metadata annotations > from checker-qual and errorprone, for example @NotNull and > @CanIgnoreReturnValue. The module looks like this: > module com.github.benmanes.caffeine { > exports com.github.benmanes.caffeine.cache; > exports com.github.benmanes.caffeine.cache.stats; > > requires static transitive com.google.errorprone.annotations; > requires static transitive org.checkerframework.checker.qual; > } > > The annotations are not required at runtime, hence static. They're > visibly placed on public methods and return types, so that API clients > can benefit from them for the purposes of annotation-based null > analysis, kotlin interop, etc. As the annotations are part of the API, > they're marked transitive. > > However, the "transitive" aspect imposes some requirements on users. I > am wondering if there is a more correct way to declare these > annotation dependencies than static transitive. > > One user would like to avoid the presence of these annotations at > compile-time. For reference, here's the relevant discussion: > https://github.com/ben-manes/caffeine/issues/535 > > I'm not a maintainer of caffeine, though I was involved in its modularization. > > Regards, > Anand > From anandebeh at gmail.com Mon Jun 7 11:45:59 2021 From: anandebeh at gmail.com (Anand Beh) Date: Mon, 7 Jun 2021 07:45:59 -0400 Subject: Annotation Dependencies and Requires Static Transitive In-Reply-To: <353f78cf-6e41-9906-b3d3-ffbf76868b14@oracle.com> References: <353f78cf-6e41-9906-b3d3-ffbf76868b14@oracle.com> Message-ID: Thanks to you both for the advice. Following on your suggestions, Caffeine changed to "requires static". However, javac now produces a warning about the lack of transitivity: warning: [exports] class Nullable in module org.checkerframework.checker.qual is not indirectly exported using requires transitive Is this a bug in javac or is it anything we should be worried about? Regards, Anand On 6/3/21, Alex Buckley wrote: > Even without `transitive`, requiring modules with `static` means that > anyone who habitually builds their entire stack from source will still > need the errorprone and checker-qual modules at compile time. > > There are no "run-time only" dependencies in module declarations, unless > services come into play, which is not realistic here. As Remi said, > `requires static` is your best bet. Annotation types that are exported > by a third party such as Google, in order for people to annotate their > personal codebases, are not really an API that those personal codebases > need to export -- any fourth party program that wishes to inspect the > annotations in your codebase needs to arrange its own dependency on the > annotation types from the third party. > > Alex > > On 6/3/2021 1:10 PM, Anand Beh wrote: >> Hello, >> >> The cache library Caffeine recently added a full module descriptor. It >> has no runtime dependencies, but it depends on metadata annotations >> from checker-qual and errorprone, for example @NotNull and >> @CanIgnoreReturnValue. The module looks like this: >> module com.github.benmanes.caffeine { >> exports com.github.benmanes.caffeine.cache; >> exports com.github.benmanes.caffeine.cache.stats; >> >> requires static transitive com.google.errorprone.annotations; >> requires static transitive org.checkerframework.checker.qual; >> } >> >> The annotations are not required at runtime, hence static. They're >> visibly placed on public methods and return types, so that API clients >> can benefit from them for the purposes of annotation-based null >> analysis, kotlin interop, etc. As the annotations are part of the API, >> they're marked transitive. >> >> However, the "transitive" aspect imposes some requirements on users. I >> am wondering if there is a more correct way to declare these >> annotation dependencies than static transitive. >> >> One user would like to avoid the presence of these annotations at >> compile-time. For reference, here's the relevant discussion: >> https://github.com/ben-manes/caffeine/issues/535 >> >> I'm not a maintainer of caffeine, though I was involved in its >> modularization. >> >> Regards, >> Anand >> > From Alexey1.Gavrilov at gmail.com Tue Jun 8 19:20:29 2021 From: Alexey1.Gavrilov at gmail.com (Alexey Gavrilov) Date: Tue, 8 Jun 2021 21:20:29 +0200 Subject: Module resource loading Message-ID: Hi, While working on modularization of my application I've noticed encapsulation rules for non-class-file resources are different depending on the resource directory names. It was not quite obvious for me and my colleagues even though it is a part of the spec. I wonder if this can be possibly clarified in the documentation. Please see an example below. The specification for [getResourceAsStream][1] says that the resource is encapsulated if a *package name* is derived from its name. So if the [resource?s directory name is NOT a valid Java identifier][2], it is NOT encapsulated. Which means that if a module has a resource located under, for example, a directory named `dir-3` (containing a non-valid character `-` in its name) it will always be accessible from outside of the module. Here is an example of two Java modules ([source code in GitHub][3]). *Module 1* consists of the following resource files: ??? dir-3 ? ??? resource3.txt ??? dir1 ? ??? resource1.txt ??? dir2 ? ??? resource2.txt ??? root.txt and `module-info.java`: module module_one { opens dir1; } *Module 2* requires *Module 1* in `module-info.java`: module module_two { requires module_one; } and has a sample main class for loading various resource files: package module2; import java.io.IOException; public class Main { public static void main(String[] args) throws IOException { loadResource("root.txt", "From module's root directory"); loadResource("dir1/resource1.txt", "From opened package `dir1`"); loadResource("dir2/resource2.txt", "From internal package `dir2`"); loadResource("dir-3/resource3.txt", "From directory `dir-3` with non-Java name"); } public static void loadResource(String name, String comment) throws IOException { // module2 application class loader final var classLoader = Main.class.getClassLoader(); try (var in = classLoader.getResourceAsStream(name)) { System.out.println(); System.out.println("// " + comment); System.out.println(name + ": " + (in != null)); } } } Running the code above gives the following output: // From module's root directory root.txt: true // From opened package `dir1` dir1/resource1.txt: true // From internal package `dir2` dir2/resource2.txt: false // From directory `dir-3` with non-Java name dir-3/resource3.txt: true As you can see the resource file from the root directory and from the `dir-3` directory are not encapsulated, therefore *Module 2* can load them. The package `dir1` is encapsulated but unconditionally opened. *Module 2* can load it as well. The package `dir2` is encapsulated and not opened. *Module 2* cannot load it. Note that *Module 2* cannot contain its own resources under `dir1` and `dir2` directories because they are already encapsulated in *Module 1*. If you try adding `dir1` you will get the following error: Error occurred during initialization of boot layer java.lang.LayerInstantiationException: Package dir1 in both module module_one and module module_two Regards, Alexey [1]: https://docs.oracle.com/javase/9/docs/api/java/lang/Module.html#getResourceAsStream-java.lang.String- [2]: https://docs.oracle.com/javase/8/docs/technotes/guides/lang/resources.html [3]: https://github.com/agavrilov76/jpms-resource From Alan.Bateman at oracle.com Wed Jun 9 15:08:30 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 9 Jun 2021 16:08:30 +0100 Subject: Module resource loading In-Reply-To: References: Message-ID: On 08/06/2021 20:20, Alexey Gavrilov wrote: > Note that *Module 2* cannot contain its own resources under `dir1` and > `dir2` directories because they are already encapsulated in *Module 1*. If > you try adding `dir1` you will get the following error: > > Error occurred during initialization of boot layer > java.lang.LayerInstantiationException: Package dir1 in both module > module_one and module module_two Resource encapsulation is complicated. The basic guideline is that code should use Class::getResourceXXX or Module::getResourceXXX to locate a resource in its own module. ClassLoader::getResourceXXXX is for locating other resources in other components on the class path or module path. Services are often a better choice for cases where the resource is configuration that names a class. Two modules with resources in the same "package" is the same as split package issue. Are the two modules in question closely related, meaning sharing a name space, or they are completely unrelated? -Alan. From Alexey1.Gavrilov at gmail.com Wed Jun 9 22:44:47 2021 From: Alexey1.Gavrilov at gmail.com (Alexey Gavrilov) Date: Thu, 10 Jun 2021 00:44:47 +0200 Subject: Module resource loading In-Reply-To: References: Message-ID: Thanks for taking a look. > Resource encapsulation is complicated. Yes, that is very true In our application we use the TypeSafe?s config library [1] which scans the application classpath for all resources named like `reference.conf` or `application.conf`. It works just fine in a modular application, the config files are in the root directory and not encapsulated. No problems. We also use the Flyway database migration tool [2] which looks for .sql files located under `db/migration` directories in the classpath. We used to have those directories in different modules, which is frankly not a good idea, Flyway was not able to find the migration .sql files because of the package split. We also have some migration files under `db/migration-` directories and noticed that it works differently because of a non-valid Java character in the directory name. Hence Flyway is able to load the files located in different modules. And finally our JUnit tests for the .sql migrations are running in the classpath mode, making it hard to debug. Based on this experience I?ve created a Flyway feature request [3] for Java Modules support. [1] https://github.com/lightbend/config [2] https://flywaydb.org [3] https://github.com/flyway/flyway/issues/3196 Best regards, -Alexey On Wed, Jun 9, 2021 at 5:08 PM Alan Bateman wrote: > On 08/06/2021 20:20, Alexey Gavrilov wrote: > > Note that *Module 2* cannot contain its own resources under `dir1` and > > `dir2` directories because they are already encapsulated in *Module 1*. > If > > you try adding `dir1` you will get the following error: > > > > Error occurred during initialization of boot layer > > java.lang.LayerInstantiationException: Package dir1 in both module > > module_one and module module_two > Resource encapsulation is complicated. > > The basic guideline is that code should use Class::getResourceXXX or > Module::getResourceXXX to locate a resource in its own module. > ClassLoader::getResourceXXXX is for locating other resources in other > components on the class path or module path. Services are often a better > choice for cases where the resource is configuration that names a class. > > Two modules with resources in the same "package" is the same as split > package issue. Are the two modules in question closely related, meaning > sharing a name space, or they are completely unrelated? > > -Alan. > From christian.beikov at gmail.com Mon Jun 28 08:49:18 2021 From: christian.beikov at gmail.com (Christian Beikov) Date: Mon, 28 Jun 2021 10:49:18 +0200 Subject: Requires runtime considered? Message-ID: <481fbd25-be3b-30fe-34a3-0a4f2d035dbb@gmail.com> Hello experts, I am in the process of modularizing one of my projects and hit some rather annoying limitations that requires me to change parts of my build and dependencies due to the way the Java module system works. I usually have modules like x-api, x-impl and x-testsuite where I have a compile time dependency from x-impl to x-api. In x-testsuite I only code against x-api, so I define a Maven runtime dependency on x-impl in order for my CDI container to pick up the implementations. The annoying part is that with modularization I now need to declare my dependency in x-testsuite on x-impl as compile time dependency, because otherwise I can't list the module in the module-info. If I don't list the module, then it is not "visible" to the CDI container, which I guess is on purpose, even though it probably ends up in the module path. Is that by design that classes are not visible if no module depends on a module like x-impl in such a scenario, or is that maybe something that could work and should be improved in the CDI implementation? I was thinking that if "requires runtime ..." were allowed without checking that the module is available during compilation, this would work fine. Did this idea come up before already? Regards, Christian From Alan.Bateman at oracle.com Mon Jun 28 09:21:08 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 28 Jun 2021 10:21:08 +0100 Subject: Requires runtime considered? In-Reply-To: <481fbd25-be3b-30fe-34a3-0a4f2d035dbb@gmail.com> References: <481fbd25-be3b-30fe-34a3-0a4f2d035dbb@gmail.com> Message-ID: On 28/06/2021 09:49, Christian Beikov wrote: > Hello experts, > > I am in the process of modularizing one of my projects and hit some > rather annoying limitations that requires me to change parts of my > build and dependencies due to the way the Java module system works. > > I usually have modules like x-api, x-impl and x-testsuite where I have > a compile time dependency from x-impl to x-api. In x-testsuite I only > code against x-api, so I define a Maven runtime dependency on x-impl > in order for my CDI container to pick up the implementations. The > annoying part is that with modularization I now need to declare my > dependency in x-testsuite on x-impl as compile time dependency, > because otherwise I can't list the module in the module-info. If I > don't list the module, then it is not "visible" to the CDI container, > which I guess is on purpose, even though it probably ends up in the > module path. > > Is that by design that classes are not visible if no module depends on > a module like x-impl in such a scenario, or is that maybe something > that could work and should be improved in the CDI implementation? I > was thinking that if "requires runtime ..." were allowed without > checking that the module is available during compilation, this would > work fine. Did this idea come up before already? The module system supports services. x-api uses a service, x-impl provides an implementation of the service. If x-api is resolved then all observable modules that provide an implementation of the services used by x-api will be resolved too. In this scenario, x-impl requires x-api, x-api does not require x-impl. You used the phrase "compile time dependency" but it's more than that, it's all phases as there will be references in x-impl to x-api's exported API. I don't know if you control the CDI implementation or not but if you aren't using services then you'll need to help the module system with "--add-modules x-impl". This is because no other module requires x-impl and it's not providing an implementation of a service. It may be observable but it's the same directory as 99,999 other modules that you don't care about. -Alan From christian.beikov at gmail.com Mon Jun 28 09:40:05 2021 From: christian.beikov at gmail.com (Christian Beikov) Date: Mon, 28 Jun 2021 11:40:05 +0200 Subject: Requires runtime considered? In-Reply-To: References: <481fbd25-be3b-30fe-34a3-0a4f2d035dbb@gmail.com> Message-ID: <0c2d4101-fb80-a82f-3901-e5cf1e09c26a@gmail.com> Thanks for the quick answer and the hint about service, but as you can imagine, services don't really help here since I am trying to use CDI which does it's own discovery. I understand the point about "99,999 other modules", that's why I asked specifically if such a module is even "visible" somehow such that a CDI container could do something about this. As far as I know, the Maven surefire plugin adds only the module of the artifact for which to execute tests to the module path via --add-modules. Maybe I should request that runtime dependencies should be added with the --add-modules flag as well. This sounds like a promising solution though, thanks for the tip! Regards, Christian Am 28.06.2021 um 11:21 schrieb Alan Bateman: > On 28/06/2021 09:49, Christian Beikov wrote: >> Hello experts, >> >> I am in the process of modularizing one of my projects and hit some >> rather annoying limitations that requires me to change parts of my >> build and dependencies due to the way the Java module system works. >> >> I usually have modules like x-api, x-impl and x-testsuite where I >> have a compile time dependency from x-impl to x-api. In x-testsuite I >> only code against x-api, so I define a Maven runtime dependency on >> x-impl in order for my CDI container to pick up the implementations. >> The annoying part is that with modularization I now need to declare >> my dependency in x-testsuite on x-impl as compile time dependency, >> because otherwise I can't list the module in the module-info. If I >> don't list the module, then it is not "visible" to the CDI container, >> which I guess is on purpose, even though it probably ends up in the >> module path. >> >> Is that by design that classes are not visible if no module depends >> on a module like x-impl in such a scenario, or is that maybe >> something that could work and should be improved in the CDI >> implementation? I was thinking that if "requires runtime ..." were >> allowed without checking that the module is available during >> compilation, this would work fine. Did this idea come up before already? > > The module system supports services. x-api uses a service, x-impl > provides an implementation of the service. If x-api is resolved then > all observable modules that provide an implementation of the services > used by x-api will be resolved too. In this scenario, x-impl requires > x-api, x-api does not require x-impl. You used the phrase "compile > time dependency" but it's more than that, it's all phases as there > will be references in x-impl to x-api's exported API. > > I don't know if you control the CDI implementation or not but if you > aren't using services then you'll need to help the module system with > "--add-modules x-impl". This is because no other module requires > x-impl and it's not providing an implementation of a service. It may > be observable but it's the same directory as 99,999 other modules that > you don't care about. > > -Alan