From frost.gary at gmail.com Mon Apr 20 09:17:08 2020 From: frost.gary at gmail.com (Gary Frost) Date: Mon, 20 Apr 2020 10:17:08 +0100 Subject: BackEnd Service Provider. Message-ID: I feel I am missing something obvious ;) I am working (well playing actually) with Graal and JDK13 and have built my own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to inject some custom code into generated x86. Very cool. If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory class (from JDK13 source) I can instantiate my factory directly, and I see my modified code. But of course I want to able to build against and use use my factory using a clean opendjdk JDK13 binary distribution. It looks like the class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to allow me to offer my factory as a 'service provider' which the CompilerConfigurationFactory accesses via this call GraalServices.load(HotSpotBackendFactory.class) but nothing I try works. It looks like I need to provide a module-info.java class, but of course I only want my one service to override the default. Any suggestions/help oir pointers to docs would be appreciated. Gary From doug.simon at oracle.com Mon Apr 20 10:53:32 2020 From: doug.simon at oracle.com (Doug Simon) Date: Mon, 20 Apr 2020 12:53:32 +0200 Subject: BackEnd Service Provider. In-Reply-To: References: Message-ID: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> Hi Gary, Have you tried creating a module for your service? I think that?s required as of JDK 9. I suggest opening a PR on https://github.com/oracle/graal so we can have a more concrete discussion. -Doug > On 20 Apr 2020, at 11:17, Gary Frost wrote: > > I feel I am missing something obvious ;) > > I am working (well playing actually) with Graal and JDK13 and have built my > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to > inject some custom code into generated x86. Very cool. > > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory > class (from JDK13 source) I can instantiate my factory directly, and I see > my modified code. > > But of course I want to able to build against and use use my factory using > a clean opendjdk JDK13 binary distribution. > > It looks like the > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to > allow me to offer my factory as a 'service provider' which the > CompilerConfigurationFactory accesses via this call > GraalServices.load(HotSpotBackendFactory.class) > but nothing I try works. > > It looks like I need to provide a module-info.java class, but of course I > only want my one service to override the default. > > Any suggestions/help oir pointers to docs would be appreciated. > > Gary From frost.gary at gmail.com Mon Apr 20 12:44:09 2020 From: frost.gary at gmail.com (Gary Frost) Date: Mon, 20 Apr 2020 13:44:09 +0100 Subject: BackEnd Service Provider. In-Reply-To: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> Message-ID: Yeah I started created a module, but it seems for my module to compile I need jdk.internal.vm.ci module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my named module. Which seems wrong. I wanted to avoid using GraalVM. I figured I should be able to do what I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler I can 'get around it' by pulling the source for org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking this one file (to load my Factory) then building using using --patch-module jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is clearly hacky. I should not have to use --patch-module here. Alas using --patch-module renders IntelliJ next to useless (it ignores patch-module) as pointed out by John Rose and myself here. https://youtrack.jetbrains.com/issue/IDEA-224342 Maybe the intent was that the JDK versions of Graal, were just to 'use Graal', not to allow developers to add there own features. Maybe that is the distiction between GraalVM and Graal enabled JDK. Gary On Mon, Apr 20, 2020 at 11:54 AM Doug Simon wrote: > Hi Gary, > > Have you tried creating a module for your service? I think that?s required > as of JDK 9. > > I suggest opening a PR on https://github.com/oracle/graal so we can have > a more concrete discussion. > > -Doug > > > On 20 Apr 2020, at 11:17, Gary Frost wrote: > > > > I feel I am missing something obvious ;) > > > > I am working (well playing actually) with Graal and JDK13 and have built > my > > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory > to > > inject some custom code into generated x86. Very cool. > > > > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory > > class (from JDK13 source) I can instantiate my factory directly, and I > see > > my modified code. > > > > But of course I want to able to build against and use use my factory > using > > a clean opendjdk JDK13 binary distribution. > > > > It looks like the > > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup > to > > allow me to offer my factory as a 'service provider' which the > > CompilerConfigurationFactory accesses via this call > > GraalServices.load(HotSpotBackendFactory.class) > > but nothing I try works. > > > > It looks like I need to provide a module-info.java class, but of course I > > only want my one service to override the default. > > > > Any suggestions/help oir pointers to docs would be appreciated. > > > > Gary > > From doug.simon at oracle.com Mon Apr 20 12:50:25 2020 From: doug.simon at oracle.com (Doug Simon) Date: Mon, 20 Apr 2020 14:50:25 +0200 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> Message-ID: <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> Hi Gary, > On 20 Apr 2020, at 14:44, Gary Frost wrote: > > Yeah I started created a module, but it seems for my module to compile I need jdk.internal.vm.ci module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my named module. Which seems wrong. Actually, that?s by design. Only standard JDK API can be exported unqualified from the JDK. Neither Graal nor JVMCI is public API and so --add-exports is needed for compilation. > > I wanted to avoid using GraalVM. I figured I should be able to do what I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler You can - you just need --add-exports (and maybe some other options) to ?open up? the classes you?re trying to hook into. > I can 'get around it' by pulling the source for org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking this one file (to load my Factory) then building using using --patch-module jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is clearly hacky. I should not have to use --patch-module here. > > Alas using --patch-module renders IntelliJ next to useless (it ignores patch-module) as pointed out by John Rose and myself here. > https://youtrack.jetbrains.com/issue/IDEA-224342 > > Maybe the intent was that the JDK versions of Graal, were just to 'use Graal', not to allow developers to add there own features. > > Maybe that is the distiction between GraalVM and Graal enabled JDK. No, both of these don?t not offer open extension points. You need to force it open on the command as mentioned above. We can?t have just any old app code getting their hands on the compiler ;-) -Doug > On Mon, Apr 20, 2020 at 11:54 AM Doug Simon > wrote: > Hi Gary, > > Have you tried creating a module for your service? I think that?s required as of JDK 9. > > I suggest opening a PR on https://github.com/oracle/graal so we can have a more concrete discussion. > > -Doug > > > On 20 Apr 2020, at 11:17, Gary Frost > wrote: > > > > I feel I am missing something obvious ;) > > > > I am working (well playing actually) with Graal and JDK13 and have built my > > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to > > inject some custom code into generated x86. Very cool. > > > > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory > > class (from JDK13 source) I can instantiate my factory directly, and I see > > my modified code. > > > > But of course I want to able to build against and use use my factory using > > a clean opendjdk JDK13 binary distribution. > > > > It looks like the > > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to > > allow me to offer my factory as a 'service provider' which the > > CompilerConfigurationFactory accesses via this call > > GraalServices.load(HotSpotBackendFactory.class) > > but nothing I try works. > > > > It looks like I need to provide a module-info.java class, but of course I > > only want my one service to override the default. > > > > Any suggestions/help oir pointers to docs would be appreciated. > > > > Gary > From frost.gary at gmail.com Tue Apr 21 10:30:42 2020 From: frost.gary at gmail.com (Gary Frost) Date: Tue, 21 Apr 2020 11:30:42 +0100 Subject: BackEnd Service Provider. In-Reply-To: <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> Message-ID: I don't want to flog this horse excessively, but please humour me one more time. Even after defining a module I can't offer my own service provider for org.graalvm.compiler.hotspot.HotSpotBackendFactory I really think we need a way for folks to experiment with Graal backends without having to build there own JDK/JVM. I created a module to be able to map my service provider src/org.grfstuff.compiler/ ??? module-info.java ??? org ??? grfstuff ??? compiler ??? GrfStuffHotSpotBackendFactory.java Where my module-info.java is this. --8<-- module org.grfstuff.compiler { requires jdk.internal.vm.compiler; provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; } -->8-- My implementation is just enough code so I can see an instance of my backend get loaded. --8<-- package org.grfstuff.compiler; import org.graalvm.compiler.hotspot.HotSpotBackendFactory; import org.graalvm.compiler.serviceprovider.ServiceProvider; import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; @ServiceProvider(HotSpotBackendFactory.class) public class GrfStuffHotSpotBackendFactory extends AMD64HotSpotBackendFactory { public GrfStuffHotSpotBackendFactory(){ System.out.println("GrfStuffHotSpotFactory"); } public String toString() { return "GRFSTUFF_AMD64"; } } -->8-- I can build my module using appropriate --add-exports 'jiggery pokery ' :) javac -g \ -d build \ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ --module-source-path src\ --module org.grfstuff.compiler\ $(shell find src/org.grfstuff.compiler -name *.java) But at runtime (my app is a compute intensive voilajones face detection algorithm) java \ -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\ -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ -p build -m org.grfstuff.compiler\ -classpath build/org.grfstuff.violajones\ org.grfstuff.violajones.ViolaJones I get this error Error occurred during initialization of boot layer java.lang.module.ResolutionException: Module org.grfstuff.compiler does not read a module that exports org.graalvm.compiler.hotspot It would appear that whilst add-exports is enough at compile time to persuade the compiler that all is well. At runtime the service loader execution path does not honour the add-exports requests to the service layer. In the end my temporary solution was to add this stanza to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's constructor. --8<-- String backendFactoryClassName = System.getProperty("graal.backend.factory"); // org.grfstuff.compiler.GrfStuffHotSpotBackendFactory if (backendFactoryClassName != null){ try { Class factoryClass = (Class) Class.forName(backendFactoryClassName); HotSpotBackendFactory factory = factoryClass.getDeclaredConstructor().newInstance(); Class arch = factory.getArchitecture(); HotSpotBackendFactory oldEntry = backends.put(arch, factory); }catch(Exception e){ } }else { //existing code } -->8-- Then I can build a jdk13 image and run using a combination of --add-modules along with -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory Whilst this works for me. I am quite sure if our common goal is to get users to 'adopt/play-with' Graal, we can't expect them to build a dedicated OpenJDK image. I think it is an oversite to offer a ServiceProvider extension mechanism (HotSpotBackendFactory) only for the module system to forbid anyone from using it. It would be great if we could offer a simpler mechanism (without resorting to --patch-module which does not seem to be supported by IntelliJ). We already have some protection by forcing -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just open this up a little more behind these current flags. Gary On Mon, Apr 20, 2020 at 1:52 PM Doug Simon wrote: > Hi Gary, > > On 20 Apr 2020, at 14:44, Gary Frost wrote: > > Yeah I started created a module, but it seems for my module to compile I > need jdk.internal.vm.ci > > module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my > named module. Which seems wrong. > > > Actually, that?s by design. Only standard JDK API can be exported > unqualified from the JDK. Neither Graal nor JVMCI is public API and so > --add-exports is needed for compilation. > > > I wanted to avoid using GraalVM. I figured I should be able to do what I > want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions > -XX:+UseJVMCICompiler > > > You can - you just need --add-exports (and maybe some other options) to > ?open up? the classes you?re trying to hook into. > > I can 'get around it' by pulling the source for > org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking > this one file (to load my Factory) then building using > using --patch-module > jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is > clearly hacky. I should not have to use --patch-module here. > > Alas using --patch-module renders IntelliJ next to useless (it ignores > patch-module) as pointed out by John Rose and myself here. > https://youtrack.jetbrains.com/issue/IDEA-224342 > > > Maybe the intent was that the JDK versions of Graal, were just to 'use > Graal', not to allow developers to add there own features. > > Maybe that is the distiction between GraalVM and Graal enabled JDK. > > > No, both of these don?t not offer open extension points. You need to force > it open on the command as mentioned above. We can?t have just any old app > code getting their hands on the compiler ;-) > > -Doug > > On Mon, Apr 20, 2020 at 11:54 AM Doug Simon wrote: > >> Hi Gary, >> >> Have you tried creating a module for your service? I think that?s >> required as of JDK 9. >> >> I suggest opening a PR on https://github.com/oracle/graal >> >> so we can have a more concrete discussion. >> >> -Doug >> >> > On 20 Apr 2020, at 11:17, Gary Frost wrote: >> > >> > I feel I am missing something obvious ;) >> > >> > I am working (well playing actually) with Graal and JDK13 and have >> built my >> > own implementation of >> org.graalvm.compiler.hotspot.HotSpotBackendFactory to >> > inject some custom code into generated x86. Very cool. >> > >> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory >> > class (from JDK13 source) I can instantiate my factory directly, and I >> see >> > my modified code. >> > >> > But of course I want to able to build against and use use my factory >> using >> > a clean opendjdk JDK13 binary distribution. >> > >> > It looks like the >> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is >> setup to >> > allow me to offer my factory as a 'service provider' which the >> > CompilerConfigurationFactory accesses via this call >> > GraalServices.load(HotSpotBackendFactory.class) >> > but nothing I try works. >> > >> > It looks like I need to provide a module-info.java class, but of course >> I >> > only want my one service to override the default. >> > >> > Any suggestions/help oir pointers to docs would be appreciated. >> > >> > Gary >> >> > From doug.simon at oracle.com Tue Apr 21 10:47:27 2020 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 21 Apr 2020 12:47:27 +0200 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> Message-ID: <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> > On 21 Apr 2020, at 12:30, Gary Frost wrote: > > I don't want to flog this horse excessively, but please humour me one more time. > > Even after defining a module I can't offer my own service provider for org.graalvm.compiler.hotspot.HotSpotBackendFactory > > I really think we need a way for folks to experiment with Graal backends without having to build there own JDK/JVM. > > I created a module to be able to map my service provider > > src/org.grfstuff.compiler/ > ??? module-info.java > ??? org > ??? grfstuff > ??? compiler > ??? GrfStuffHotSpotBackendFactory.java > > > Where my module-info.java is this. > --8<-- > module org.grfstuff.compiler { > requires jdk.internal.vm.compiler; > provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; > } > -->8-- > > My implementation is just enough code so I can see an instance of my backend get loaded. > --8<-- > package org.grfstuff.compiler; > import org.graalvm.compiler.hotspot.HotSpotBackendFactory; > import org.graalvm.compiler.serviceprovider.ServiceProvider; > import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; > > @ServiceProvider(HotSpotBackendFactory.class) > public class GrfStuffHotSpotBackendFactory extends AMD64HotSpotBackendFactory { > public GrfStuffHotSpotBackendFactory(){ > System.out.println("GrfStuffHotSpotFactory"); > } > public String toString() { > return "GRFSTUFF_AMD64"; > } > } > -->8-- > > I can build my module using appropriate --add-exports 'jiggery pokery ' :) > > javac -g \ > -d build \ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ > --module-source-path src\ > --module org.grfstuff.compiler\ > $(shell find src/org.grfstuff.compiler -name *.java) > > But at runtime (my app is a compute intensive voilajones face detection algorithm) > java \ > -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\ > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ > -p build -m org.grfstuff.compiler\ > -classpath build/org.grfstuff.violajones\ > org.grfstuff.violajones.ViolaJones > > I get this error > > Error occurred during initialization of boot layer > java.lang.module.ResolutionException: Module org.grfstuff.compiler does not read a module that exports org.graalvm.compiler.hotspot > > It would appear that whilst add-exports is enough at compile time to persuade the compiler that all is well. At runtime the service loader execution path does not honour the add-exports requests to the service layer. If you use ?add-exports during java compilation, you almost certainly need to use it at runtime. That is, repeat your --add-export arguments to the java launcher. You may also need some --add-reads options. If you put you jars up somewhere, I can help find the right incantation if you?re still having trouble. > In the end my temporary solution was to add this stanza to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's constructor. > > --8<-- > String backendFactoryClassName = System.getProperty("graal.backend.factory"); // org.grfstuff.compiler.GrfStuffHotSpotBackendFactory > if (backendFactoryClassName != null){ > try { > Class factoryClass = (Class) Class.forName(backendFactoryClassName); > HotSpotBackendFactory factory = factoryClass.getDeclaredConstructor().newInstance(); > Class arch = factory.getArchitecture(); > HotSpotBackendFactory oldEntry = backends.put(arch, factory); > }catch(Exception e){ > > } > }else { > //existing code > } > -->8-- > > Then I can build a jdk13 image and run using a combination of --add-modules along with -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory > > Whilst this works for me. I am quite sure if our common goal is to get users to 'adopt/play-with' Graal, we can't expect them to build a dedicated OpenJDK image. > > I think it is an oversite to offer a ServiceProvider extension mechanism (HotSpotBackendFactory) only for the module system to forbid anyone from using it. This is a constraint imposed on us by the fact that Graal is part of the JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think the inconvenience of adding a few --add-exports and --add-reads options to javac and java is an acceptable overhead to experiment with Graal. We just need to get the incantation right and hope someone writes a nice blog article describing it ;-) > It would be great if we could offer a simpler mechanism (without resorting to --patch-module which does not seem to be supported by IntelliJ). > > We already have some protection by forcing -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just open this up a little more behind these current flags. As stated above, the JVMCI and Graal modules cannot be opened up by default. -Doug > > > > On Mon, Apr 20, 2020 at 1:52 PM Doug Simon > wrote: > Hi Gary, > >> On 20 Apr 2020, at 14:44, Gary Frost > wrote: >> >> Yeah I started created a module, but it seems for my module to compile I need jdk.internal.vm.ci module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my named module. Which seems wrong. > > Actually, that?s by design. Only standard JDK API can be exported unqualified from the JDK. Neither Graal nor JVMCI is public API and so --add-exports is needed for compilation. > >> >> I wanted to avoid using GraalVM. I figured I should be able to do what I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler > > You can - you just need --add-exports (and maybe some other options) to ?open up? the classes you?re trying to hook into. > >> I can 'get around it' by pulling the source for org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking this one file (to load my Factory) then building using using --patch-module jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is clearly hacky. I should not have to use --patch-module here. >> >> Alas using --patch-module renders IntelliJ next to useless (it ignores patch-module) as pointed out by John Rose and myself here. >> https://youtrack.jetbrains.com/issue/IDEA-224342 >> >> Maybe the intent was that the JDK versions of Graal, were just to 'use Graal', not to allow developers to add there own features. >> >> Maybe that is the distiction between GraalVM and Graal enabled JDK. > > No, both of these don?t not offer open extension points. You need to force it open on the command as mentioned above. We can?t have just any old app code getting their hands on the compiler ;-) > > -Doug > >> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon > wrote: >> Hi Gary, >> >> Have you tried creating a module for your service? I think that?s required as of JDK 9. >> >> I suggest opening a PR on https://github.com/oracle/graal so we can have a more concrete discussion. >> >> -Doug >> >> > On 20 Apr 2020, at 11:17, Gary Frost > wrote: >> > >> > I feel I am missing something obvious ;) >> > >> > I am working (well playing actually) with Graal and JDK13 and have built my >> > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to >> > inject some custom code into generated x86. Very cool. >> > >> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory >> > class (from JDK13 source) I can instantiate my factory directly, and I see >> > my modified code. >> > >> > But of course I want to able to build against and use use my factory using >> > a clean opendjdk JDK13 binary distribution. >> > >> > It looks like the >> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to >> > allow me to offer my factory as a 'service provider' which the >> > CompilerConfigurationFactory accesses via this call >> > GraalServices.load(HotSpotBackendFactory.class) >> > but nothing I try works. >> > >> > It looks like I need to provide a module-info.java class, but of course I >> > only want my one service to override the default. >> > >> > Any suggestions/help oir pointers to docs would be appreciated. >> > >> > Gary >> > > From frost.gary at gmail.com Tue Apr 21 10:59:03 2020 From: frost.gary at gmail.com (Gary Frost) Date: Tue, 21 Apr 2020 11:59:03 +0100 Subject: BackEnd Service Provider. In-Reply-To: <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> Message-ID: I apologise Doug My argument is that there is no combination of '-add-exports and --add-reads options to javac and java' that will work here. Try it! ;) I do/did in fact have it at runtime as well. I stripped it from my description because it seems to make no difference. /usr/lib/jvm/jdk-13/bin/java \ -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\ -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ -p build -m org.grfstuff.compiler\ -classpath build/org.grfstuff.violajones\ org.grfstuff.violajones.ViolaJones I attached a zip of the project to my last email and it got stripped. My argument is that there is no combination of '-add-exports and --add-reads options to javac and java' that will work. The only option is --patch-module which is not even officially offered by javac or java and is not supported by IDE's $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l 0 $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l 0 I only know of this option because I read the source code ;) Gary On Tue, Apr 21, 2020 at 11:47 AM Doug Simon wrote: > > > On 21 Apr 2020, at 12:30, Gary Frost wrote: > > I don't want to flog this horse excessively, but please humour me one > more time. > > Even after defining a module I can't offer my own service provider for > org.graalvm.compiler.hotspot.HotSpotBackendFactory > > I really think we need a way for folks to experiment with Graal backends > without having to build there own JDK/JVM. > > I created a module to be able to map my service provider > > src/org.grfstuff.compiler/ > ??? module-info.java > ??? org > ??? grfstuff > ??? compiler > ??? GrfStuffHotSpotBackendFactory.java > > > Where my module-info.java is this. > --8<-- > module org.grfstuff.compiler { > requires jdk.internal.vm.compiler; > provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with > org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; > } > -->8-- > > My implementation is just enough code so I can see an instance of my > backend get loaded. > --8<-- > package org.grfstuff.compiler; > import org.graalvm.compiler.hotspot.HotSpotBackendFactory; > import org.graalvm.compiler.serviceprovider.ServiceProvider; > import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; > > @ServiceProvider(HotSpotBackendFactory.class) > public class GrfStuffHotSpotBackendFactory extends > AMD64HotSpotBackendFactory { > public GrfStuffHotSpotBackendFactory(){ > System.out.println("GrfStuffHotSpotFactory"); > } > public String toString() { > return "GRFSTUFF_AMD64"; > } > } > -->8-- > > I can build my module using appropriate --add-exports 'jiggery pokery ' :) > > javac -g \ > -d build \ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ > --module-source-path src\ > --module org.grfstuff.compiler\ > $(shell find src/org.grfstuff.compiler -name *.java) > > But at runtime (my app is a compute intensive voilajones face detection > algorithm) > java \ > -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler > -XX:-TieredCompilation\ > > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ > > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ > -p build -m org.grfstuff.compiler\ > -classpath build/org.grfstuff.violajones\ > org.grfstuff.violajones.ViolaJones > > I get this error > > Error occurred during initialization of boot layer > java.lang.module.ResolutionException: Module org.grfstuff.compiler does > not read a module that exports org.graalvm.compiler.hotspot > > It would appear that whilst add-exports is enough at compile time to > persuade the compiler that all is well. At runtime the service loader > execution path does not honour the add-exports requests to the service > layer. > > > If you use ?add-exports during java compilation, you almost certainly need > to use it at runtime. That is, repeat your --add-export arguments to the > java launcher. You may also need some --add-reads options. If you put you > jars up somewhere, I can help find the right incantation if you?re still > having trouble. > > In the end my temporary solution was to add this stanza > to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's > constructor. > > --8<-- > String backendFactoryClassName = > System.getProperty("graal.backend.factory"); // > org.grfstuff.compiler.GrfStuffHotSpotBackendFactory > if (backendFactoryClassName != null){ > try { > Class factoryClass = > (Class) > Class.forName(backendFactoryClassName); > HotSpotBackendFactory factory = > factoryClass.getDeclaredConstructor().newInstance(); > Class arch = > factory.getArchitecture(); > HotSpotBackendFactory oldEntry = > backends.put(arch, factory); > }catch(Exception e){ > > } > }else { > //existing code > } > -->8-- > > Then I can build a jdk13 image and run using a combination of > --add-modules along with > -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory > > Whilst this works for me. I am quite sure if our common goal is to get > users to 'adopt/play-with' Graal, we can't expect them to build a dedicated > OpenJDK image. > > I think it is an oversite to offer a ServiceProvider extension mechanism > (HotSpotBackendFactory) only for the module system to forbid anyone from > using it. > > > This is a constraint imposed on us by the fact that Graal is part of the > JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think > the inconvenience of adding a few --add-exports and --add-reads options to > javac and java is an acceptable overhead to experiment with Graal. We just > need to get the incantation right and hope someone writes a nice blog > article describing it ;-) > > It would be great if we could offer a simpler mechanism (without resorting > to --patch-module which does not seem to be supported by IntelliJ). > > We already have some protection by forcing > -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just > open this up a little more behind these current flags. > > > As stated above, the JVMCI and Graal modules cannot be opened up by > default. > > -Doug > > > > On Mon, Apr 20, 2020 at 1:52 PM Doug Simon wrote: > >> Hi Gary, >> >> On 20 Apr 2020, at 14:44, Gary Frost wrote: >> >> Yeah I started created a module, but it seems for my module to compile I >> need jdk.internal.vm.ci >> >> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to >> my named module. Which seems wrong. >> >> >> Actually, that?s by design. Only standard JDK API can be exported >> unqualified from the JDK. Neither Graal nor JVMCI is public API and so >> --add-exports is needed for compilation. >> >> >> I wanted to avoid using GraalVM. I figured I should be able to do what I >> want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions >> -XX:+UseJVMCICompiler >> >> >> You can - you just need --add-exports (and maybe some other options) to >> ?open up? the classes you?re trying to hook into. >> >> I can 'get around it' by pulling the source for >> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking >> this one file (to load my Factory) then building using >> using --patch-module >> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is >> clearly hacky. I should not have to use --patch-module here. >> >> Alas using --patch-module renders IntelliJ next to useless (it ignores >> patch-module) as pointed out by John Rose and myself here. >> https://youtrack.jetbrains.com/issue/IDEA-224342 >> >> >> Maybe the intent was that the JDK versions of Graal, were just to 'use >> Graal', not to allow developers to add there own features. >> >> Maybe that is the distiction between GraalVM and Graal enabled JDK. >> >> >> No, both of these don?t not offer open extension points. You need to >> force it open on the command as mentioned above. We can?t have just any old >> app code getting their hands on the compiler ;-) >> >> -Doug >> >> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon >> wrote: >> >>> Hi Gary, >>> >>> Have you tried creating a module for your service? I think that?s >>> required as of JDK 9. >>> >>> I suggest opening a PR on https://github.com/oracle/graal >>> >>> so we can have a more concrete discussion. >>> >>> -Doug >>> >>> > On 20 Apr 2020, at 11:17, Gary Frost wrote: >>> > >>> > I feel I am missing something obvious ;) >>> > >>> > I am working (well playing actually) with Graal and JDK13 and have >>> built my >>> > own implementation of >>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>> > inject some custom code into generated x86. Very cool. >>> > >>> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory >>> > class (from JDK13 source) I can instantiate my factory directly, and >>> I see >>> > my modified code. >>> > >>> > But of course I want to able to build against and use use my factory >>> using >>> > a clean opendjdk JDK13 binary distribution. >>> > >>> > It looks like the >>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is >>> setup to >>> > allow me to offer my factory as a 'service provider' which the >>> > CompilerConfigurationFactory accesses via this call >>> > GraalServices.load(HotSpotBackendFactory.class) >>> > but nothing I try works. >>> > >>> > It looks like I need to provide a module-info.java class, but of >>> course I >>> > only want my one service to override the default. >>> > >>> > Any suggestions/help oir pointers to docs would be appreciated. >>> > >>> > Gary >>> >>> >> > > > From frost.gary at gmail.com Tue Apr 21 11:23:58 2020 From: frost.gary at gmail.com (Gary Frost) Date: Tue, 21 Apr 2020 12:23:58 +0100 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> Message-ID: Here is a public github project https://github.com/grfrost/graal_service_issue git clone https://github.com/grfrost/graal_service_issue.git cd graal_service_issue # vi Makefile and set path to your JDK13 (JDK11 may work... I know some API's have changed between 11 and 13) make Gary On Tue, Apr 21, 2020 at 11:59 AM Gary Frost wrote: > I apologise Doug > > My argument is that there is no combination of '-add-exports and > --add-reads options to javac and java' that will work here. Try it! ;) > > I do/did in fact have it at runtime as well. I stripped it from my > description because it seems to make no difference. > > /usr/lib/jvm/jdk-13/bin/java \ > -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler > -XX:-TieredCompilation\ > > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ > > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ > --add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ > -p build -m org.grfstuff.compiler\ > -classpath build/org.grfstuff.violajones\ > org.grfstuff.violajones.ViolaJones > > I attached a zip of the project to my last email and it got stripped. > > My argument is that there is no combination of '-add-exports and > --add-reads options to javac and java' that will work. > > The only option is --patch-module which is not even officially offered by > javac or java and is not supported by IDE's > > $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l > 0 > $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l > 0 > > I only know of this option because I read the source code ;) > > Gary > > On Tue, Apr 21, 2020 at 11:47 AM Doug Simon wrote: > >> >> >> On 21 Apr 2020, at 12:30, Gary Frost wrote: >> >> I don't want to flog this horse excessively, but please humour me one >> more time. >> >> Even after defining a module I can't offer my own service provider for >> org.graalvm.compiler.hotspot.HotSpotBackendFactory >> >> I really think we need a way for folks to experiment with Graal backends >> without having to build there own JDK/JVM. >> >> I created a module to be able to map my service provider >> >> src/org.grfstuff.compiler/ >> ??? module-info.java >> ??? org >> ??? grfstuff >> ??? compiler >> ??? GrfStuffHotSpotBackendFactory.java >> >> >> Where my module-info.java is this. >> --8<-- >> module org.grfstuff.compiler { >> requires jdk.internal.vm.compiler; >> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with >> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; >> } >> -->8-- >> >> My implementation is just enough code so I can see an instance of my >> backend get loaded. >> --8<-- >> package org.grfstuff.compiler; >> import org.graalvm.compiler.hotspot.HotSpotBackendFactory; >> import org.graalvm.compiler.serviceprovider.ServiceProvider; >> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; >> >> @ServiceProvider(HotSpotBackendFactory.class) >> public class GrfStuffHotSpotBackendFactory extends >> AMD64HotSpotBackendFactory { >> public GrfStuffHotSpotBackendFactory(){ >> System.out.println("GrfStuffHotSpotFactory"); >> } >> public String toString() { >> return "GRFSTUFF_AMD64"; >> } >> } >> -->8-- >> >> I can build my module using appropriate --add-exports 'jiggery pokery ' :) >> >> javac -g \ >> -d build \ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >> --module-source-path src\ >> --module org.grfstuff.compiler\ >> $(shell find src/org.grfstuff.compiler -name *.java) >> >> But at runtime (my app is a compute intensive voilajones face detection >> algorithm) >> java \ >> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >> -XX:-TieredCompilation\ >> >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >> >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >> -p build -m org.grfstuff.compiler\ >> -classpath build/org.grfstuff.violajones\ >> org.grfstuff.violajones.ViolaJones >> >> I get this error >> >> Error occurred during initialization of boot layer >> java.lang.module.ResolutionException: Module org.grfstuff.compiler does >> not read a module that exports org.graalvm.compiler.hotspot >> >> It would appear that whilst add-exports is enough at compile time to >> persuade the compiler that all is well. At runtime the service loader >> execution path does not honour the add-exports requests to the service >> layer. >> >> >> If you use ?add-exports during java compilation, you almost certainly >> need to use it at runtime. That is, repeat your --add-export arguments to >> the java launcher. You may also need some --add-reads options. If you put >> you jars up somewhere, I can help find the right incantation if you?re >> still having trouble. >> >> In the end my temporary solution was to add this stanza >> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's >> constructor. >> >> --8<-- >> String backendFactoryClassName = >> System.getProperty("graal.backend.factory"); // >> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >> if (backendFactoryClassName != null){ >> try { >> Class factoryClass >> = (Class) >> Class.forName(backendFactoryClassName); >> HotSpotBackendFactory factory = >> factoryClass.getDeclaredConstructor().newInstance(); >> Class arch = >> factory.getArchitecture(); >> HotSpotBackendFactory oldEntry = >> backends.put(arch, factory); >> }catch(Exception e){ >> >> } >> }else { >> //existing code >> } >> -->8-- >> >> Then I can build a jdk13 image and run using a combination of >> --add-modules along with >> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >> >> Whilst this works for me. I am quite sure if our common goal is to get >> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated >> OpenJDK image. >> >> I think it is an oversite to offer a ServiceProvider extension mechanism >> (HotSpotBackendFactory) only for the module system to forbid anyone from >> using it. >> >> >> This is a constraint imposed on us by the fact that Graal is part of the >> JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think >> the inconvenience of adding a few --add-exports and --add-reads options to >> javac and java is an acceptable overhead to experiment with Graal. We just >> need to get the incantation right and hope someone writes a nice blog >> article describing it ;-) >> >> It would be great if we could offer a simpler mechanism (without >> resorting to --patch-module which does not seem to be supported by >> IntelliJ). >> >> We already have some protection by forcing >> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just >> open this up a little more behind these current flags. >> >> >> As stated above, the JVMCI and Graal modules cannot be opened up by >> default. >> >> -Doug >> >> >> >> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon wrote: >> >>> Hi Gary, >>> >>> On 20 Apr 2020, at 14:44, Gary Frost wrote: >>> >>> Yeah I started created a module, but it seems for my module to compile I >>> need jdk.internal.vm.ci >>> >>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>> my named module. Which seems wrong. >>> >>> >>> Actually, that?s by design. Only standard JDK API can be exported >>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so >>> --add-exports is needed for compilation. >>> >>> >>> I wanted to avoid using GraalVM. I figured I should be able to do what >>> I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions >>> -XX:+UseJVMCICompiler >>> >>> >>> You can - you just need --add-exports (and maybe some other options) to >>> ?open up? the classes you?re trying to hook into. >>> >>> I can 'get around it' by pulling the source for >>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking >>> this one file (to load my Factory) then building using >>> using --patch-module >>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is >>> clearly hacky. I should not have to use --patch-module here. >>> >>> Alas using --patch-module renders IntelliJ next to useless (it ignores >>> patch-module) as pointed out by John Rose and myself here. >>> https://youtrack.jetbrains.com/issue/IDEA-224342 >>> >>> >>> Maybe the intent was that the JDK versions of Graal, were just to 'use >>> Graal', not to allow developers to add there own features. >>> >>> Maybe that is the distiction between GraalVM and Graal enabled JDK. >>> >>> >>> No, both of these don?t not offer open extension points. You need to >>> force it open on the command as mentioned above. We can?t have just any old >>> app code getting their hands on the compiler ;-) >>> >>> -Doug >>> >>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon >>> wrote: >>> >>>> Hi Gary, >>>> >>>> Have you tried creating a module for your service? I think that?s >>>> required as of JDK 9. >>>> >>>> I suggest opening a PR on https://github.com/oracle/graal >>>> >>>> so we can have a more concrete discussion. >>>> >>>> -Doug >>>> >>>> > On 20 Apr 2020, at 11:17, Gary Frost wrote: >>>> > >>>> > I feel I am missing something obvious ;) >>>> > >>>> > I am working (well playing actually) with Graal and JDK13 and have >>>> built my >>>> > own implementation of >>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>>> > inject some custom code into generated x86. Very cool. >>>> > >>>> > If I hack the >>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory >>>> > class (from JDK13 source) I can instantiate my factory directly, and >>>> I see >>>> > my modified code. >>>> > >>>> > But of course I want to able to build against and use use my factory >>>> using >>>> > a clean opendjdk JDK13 binary distribution. >>>> > >>>> > It looks like the >>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is >>>> setup to >>>> > allow me to offer my factory as a 'service provider' which the >>>> > CompilerConfigurationFactory accesses via this call >>>> > GraalServices.load(HotSpotBackendFactory.class) >>>> > but nothing I try works. >>>> > >>>> > It looks like I need to provide a module-info.java class, but of >>>> course I >>>> > only want my one service to override the default. >>>> > >>>> > Any suggestions/help oir pointers to docs would be appreciated. >>>> > >>>> > Gary >>>> >>>> >>> >> >> >> From frost.gary at gmail.com Tue Apr 21 12:37:13 2020 From: frost.gary at gmail.com (Gary Frost) Date: Tue, 21 Apr 2020 13:37:13 +0100 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> Message-ID: I also added a makefile which uses patch-module which works. So git clone https://github.com/grfrost/graal_service_issue.git cd graal_service_issue # vi makefile* and set path to your JDK13 (JDK11 may work... I know some API's have changed between 11 and 13) make -f makefile-using-module-service-provider make -f makefile-with-patch-module On Tue, Apr 21, 2020 at 12:23 PM Gary Frost wrote: > Here is a public github project > https://github.com/grfrost/graal_service_issue > > git clone https://github.com/grfrost/graal_service_issue.git > cd graal_service_issue > # vi Makefile and set path to your JDK13 (JDK11 may work... I know some > API's have changed between 11 and 13) > make > > Gary > > > > > On Tue, Apr 21, 2020 at 11:59 AM Gary Frost wrote: > >> I apologise Doug >> >> My argument is that there is no combination of '-add-exports and >> --add-reads options to javac and java' that will work here. Try it! ;) >> >> I do/did in fact have it at runtime as well. I stripped it from my >> description because it seems to make no difference. >> >> /usr/lib/jvm/jdk-13/bin/java \ >> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >> -XX:-TieredCompilation\ >> >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >> >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >> --add-exports >> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >> -p build -m org.grfstuff.compiler\ >> -classpath build/org.grfstuff.violajones\ >> org.grfstuff.violajones.ViolaJones >> >> I attached a zip of the project to my last email and it got stripped. >> >> My argument is that there is no combination of '-add-exports and >> --add-reads options to javac and java' that will work. >> >> The only option is --patch-module which is not even officially offered by >> javac or java and is not supported by IDE's >> >> $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l >> 0 >> $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l >> 0 >> >> I only know of this option because I read the source code ;) >> >> Gary >> >> On Tue, Apr 21, 2020 at 11:47 AM Doug Simon >> wrote: >> >>> >>> >>> On 21 Apr 2020, at 12:30, Gary Frost wrote: >>> >>> I don't want to flog this horse excessively, but please humour me one >>> more time. >>> >>> Even after defining a module I can't offer my own service provider for >>> org.graalvm.compiler.hotspot.HotSpotBackendFactory >>> >>> I really think we need a way for folks to experiment with Graal backends >>> without having to build there own JDK/JVM. >>> >>> I created a module to be able to map my service provider >>> >>> src/org.grfstuff.compiler/ >>> ??? module-info.java >>> ??? org >>> ??? grfstuff >>> ??? compiler >>> ??? GrfStuffHotSpotBackendFactory.java >>> >>> >>> Where my module-info.java is this. >>> --8<-- >>> module org.grfstuff.compiler { >>> requires jdk.internal.vm.compiler; >>> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with >>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; >>> } >>> -->8-- >>> >>> My implementation is just enough code so I can see an instance of my >>> backend get loaded. >>> --8<-- >>> package org.grfstuff.compiler; >>> import org.graalvm.compiler.hotspot.HotSpotBackendFactory; >>> import org.graalvm.compiler.serviceprovider.ServiceProvider; >>> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; >>> >>> @ServiceProvider(HotSpotBackendFactory.class) >>> public class GrfStuffHotSpotBackendFactory extends >>> AMD64HotSpotBackendFactory { >>> public GrfStuffHotSpotBackendFactory(){ >>> System.out.println("GrfStuffHotSpotFactory"); >>> } >>> public String toString() { >>> return "GRFSTUFF_AMD64"; >>> } >>> } >>> -->8-- >>> >>> I can build my module using appropriate --add-exports 'jiggery pokery ' >>> :) >>> >>> javac -g \ >>> -d build \ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >>> --module-source-path src\ >>> --module org.grfstuff.compiler\ >>> $(shell find src/org.grfstuff.compiler -name *.java) >>> >>> But at runtime (my app is a compute intensive voilajones face detection >>> algorithm) >>> java \ >>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >>> -XX:-TieredCompilation\ >>> >>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >>> >>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >>> -p build -m org.grfstuff.compiler\ >>> -classpath build/org.grfstuff.violajones\ >>> org.grfstuff.violajones.ViolaJones >>> >>> I get this error >>> >>> Error occurred during initialization of boot layer >>> java.lang.module.ResolutionException: Module org.grfstuff.compiler does >>> not read a module that exports org.graalvm.compiler.hotspot >>> >>> It would appear that whilst add-exports is enough at compile time to >>> persuade the compiler that all is well. At runtime the service loader >>> execution path does not honour the add-exports requests to the service >>> layer. >>> >>> >>> If you use ?add-exports during java compilation, you almost certainly >>> need to use it at runtime. That is, repeat your --add-export arguments to >>> the java launcher. You may also need some --add-reads options. If you put >>> you jars up somewhere, I can help find the right incantation if you?re >>> still having trouble. >>> >>> In the end my temporary solution was to add this stanza >>> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's >>> constructor. >>> >>> --8<-- >>> String backendFactoryClassName = >>> System.getProperty("graal.backend.factory"); // >>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >>> if (backendFactoryClassName != null){ >>> try { >>> Class factoryClass >>> = (Class) >>> Class.forName(backendFactoryClassName); >>> HotSpotBackendFactory factory = >>> factoryClass.getDeclaredConstructor().newInstance(); >>> Class arch = >>> factory.getArchitecture(); >>> HotSpotBackendFactory oldEntry = >>> backends.put(arch, factory); >>> }catch(Exception e){ >>> >>> } >>> }else { >>> //existing code >>> } >>> -->8-- >>> >>> Then I can build a jdk13 image and run using a combination of >>> --add-modules along with >>> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >>> >>> Whilst this works for me. I am quite sure if our common goal is to get >>> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated >>> OpenJDK image. >>> >>> I think it is an oversite to offer a ServiceProvider extension mechanism >>> (HotSpotBackendFactory) only for the module system to forbid anyone from >>> using it. >>> >>> >>> This is a constraint imposed on us by the fact that Graal is part of the >>> JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think >>> the inconvenience of adding a few --add-exports and --add-reads options to >>> javac and java is an acceptable overhead to experiment with Graal. We just >>> need to get the incantation right and hope someone writes a nice blog >>> article describing it ;-) >>> >>> It would be great if we could offer a simpler mechanism (without >>> resorting to --patch-module which does not seem to be supported by >>> IntelliJ). >>> >>> We already have some protection by forcing >>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just >>> open this up a little more behind these current flags. >>> >>> >>> As stated above, the JVMCI and Graal modules cannot be opened up by >>> default. >>> >>> -Doug >>> >>> >>> >>> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon >>> wrote: >>> >>>> Hi Gary, >>>> >>>> On 20 Apr 2020, at 14:44, Gary Frost wrote: >>>> >>>> Yeah I started created a module, but it seems for my module to compile >>>> I need jdk.internal.vm.ci >>>> >>>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory >>>> to my named module. Which seems wrong. >>>> >>>> >>>> Actually, that?s by design. Only standard JDK API can be exported >>>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so >>>> --add-exports is needed for compilation. >>>> >>>> >>>> I wanted to avoid using GraalVM. I figured I should be able to do what >>>> I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions >>>> -XX:+UseJVMCICompiler >>>> >>>> >>>> You can - you just need --add-exports (and maybe some other options) to >>>> ?open up? the classes you?re trying to hook into. >>>> >>>> I can 'get around it' by pulling the source for >>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking >>>> this one file (to load my Factory) then building using >>>> using --patch-module >>>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is >>>> clearly hacky. I should not have to use --patch-module here. >>>> >>>> Alas using --patch-module renders IntelliJ next to useless (it ignores >>>> patch-module) as pointed out by John Rose and myself here. >>>> https://youtrack.jetbrains.com/issue/IDEA-224342 >>>> >>>> >>>> Maybe the intent was that the JDK versions of Graal, were just to 'use >>>> Graal', not to allow developers to add there own features. >>>> >>>> Maybe that is the distiction between GraalVM and Graal enabled JDK. >>>> >>>> >>>> No, both of these don?t not offer open extension points. You need to >>>> force it open on the command as mentioned above. We can?t have just any old >>>> app code getting their hands on the compiler ;-) >>>> >>>> -Doug >>>> >>>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon >>>> wrote: >>>> >>>>> Hi Gary, >>>>> >>>>> Have you tried creating a module for your service? I think that?s >>>>> required as of JDK 9. >>>>> >>>>> I suggest opening a PR on https://github.com/oracle/graal >>>>> >>>>> so we can have a more concrete discussion. >>>>> >>>>> -Doug >>>>> >>>>> > On 20 Apr 2020, at 11:17, Gary Frost wrote: >>>>> > >>>>> > I feel I am missing something obvious ;) >>>>> > >>>>> > I am working (well playing actually) with Graal and JDK13 and have >>>>> built my >>>>> > own implementation of >>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>>>> > inject some custom code into generated x86. Very cool. >>>>> > >>>>> > If I hack the >>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory >>>>> > class (from JDK13 source) I can instantiate my factory directly, >>>>> and I see >>>>> > my modified code. >>>>> > >>>>> > But of course I want to able to build against and use use my factory >>>>> using >>>>> > a clean opendjdk JDK13 binary distribution. >>>>> > >>>>> > It looks like the >>>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is >>>>> setup to >>>>> > allow me to offer my factory as a 'service provider' which the >>>>> > CompilerConfigurationFactory accesses via this call >>>>> > GraalServices.load(HotSpotBackendFactory.class) >>>>> > but nothing I try works. >>>>> > >>>>> > It looks like I need to provide a module-info.java class, but of >>>>> course I >>>>> > only want my one service to override the default. >>>>> > >>>>> > Any suggestions/help oir pointers to docs would be appreciated. >>>>> > >>>>> > Gary >>>>> >>>>> >>>> >>> >>> >>> From doug.simon at oracle.com Tue Apr 21 13:01:22 2020 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 21 Apr 2020 15:01:22 +0200 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> Message-ID: <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> Hi Gary, I cannot understand why --add-reads does not work and have reached out to Alan Bateman for more insight. That said, good to know that at least --patch-module works, even if it confuses IntelliJ. -Doug > On 21 Apr 2020, at 14:37, Gary Frost wrote: > > I also added a makefile which uses patch-module which works. > > So > git clone https://github.com/grfrost/graal_service_issue.git > cd graal_service_issue > # vi makefile* and set path to your JDK13 (JDK11 may work... I know some API's have changed between 11 and 13) > make -f makefile-using-module-service-provider > make -f makefile-with-patch-module > > > On Tue, Apr 21, 2020 at 12:23 PM Gary Frost > wrote: > Here is a public github project https://github.com/grfrost/graal_service_issue > > git clone https://github.com/grfrost/graal_service_issue.git > cd graal_service_issue > # vi Makefile and set path to your JDK13 (JDK11 may work... I know some API's have changed between 11 and 13) > make > > Gary > > > > > On Tue, Apr 21, 2020 at 11:59 AM Gary Frost > wrote: > I apologise Doug > > My argument is that there is no combination of '-add-exports and --add-reads options to javac and java' that will work here. Try it! ;) > > I do/did in fact have it at runtime as well. I stripped it from my description because it seems to make no difference. > > /usr/lib/jvm/jdk-13/bin/java \ > -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\ > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ > -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ > --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ > -p build -m org.grfstuff.compiler\ > -classpath build/org.grfstuff.violajones\ > org.grfstuff.violajones.ViolaJones > > I attached a zip of the project to my last email and it got stripped. > > My argument is that there is no combination of '-add-exports and --add-reads options to javac and java' that will work. > > The only option is --patch-module which is not even officially offered by javac or java and is not supported by IDE's > > $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l > 0 > $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l > 0 > > I only know of this option because I read the source code ;) > > Gary > > On Tue, Apr 21, 2020 at 11:47 AM Doug Simon > wrote: > > >> On 21 Apr 2020, at 12:30, Gary Frost > wrote: >> >> I don't want to flog this horse excessively, but please humour me one more time. >> >> Even after defining a module I can't offer my own service provider for org.graalvm.compiler.hotspot.HotSpotBackendFactory >> >> I really think we need a way for folks to experiment with Graal backends without having to build there own JDK/JVM. >> >> I created a module to be able to map my service provider >> >> src/org.grfstuff.compiler/ >> ??? module-info.java >> ??? org >> ??? grfstuff >> ??? compiler >> ??? GrfStuffHotSpotBackendFactory.java >> >> >> Where my module-info.java is this. >> --8<-- >> module org.grfstuff.compiler { >> requires jdk.internal.vm.compiler; >> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; >> } >> -->8-- >> >> My implementation is just enough code so I can see an instance of my backend get loaded. >> --8<-- >> package org.grfstuff.compiler; >> import org.graalvm.compiler.hotspot.HotSpotBackendFactory; >> import org.graalvm.compiler.serviceprovider.ServiceProvider; >> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; >> >> @ServiceProvider(HotSpotBackendFactory.class) >> public class GrfStuffHotSpotBackendFactory extends AMD64HotSpotBackendFactory { >> public GrfStuffHotSpotBackendFactory(){ >> System.out.println("GrfStuffHotSpotFactory"); >> } >> public String toString() { >> return "GRFSTUFF_AMD64"; >> } >> } >> -->8-- >> >> I can build my module using appropriate --add-exports 'jiggery pokery ' :) >> >> javac -g \ >> -d build \ >> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >> --module-source-path src\ >> --module org.grfstuff.compiler\ >> $(shell find src/org.grfstuff.compiler -name *.java) >> >> But at runtime (my app is a compute intensive voilajones face detection algorithm) >> java \ >> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\ >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >> -p build -m org.grfstuff.compiler\ >> -classpath build/org.grfstuff.violajones\ >> org.grfstuff.violajones.ViolaJones >> >> I get this error >> >> Error occurred during initialization of boot layer >> java.lang.module.ResolutionException: Module org.grfstuff.compiler does not read a module that exports org.graalvm.compiler.hotspot >> >> It would appear that whilst add-exports is enough at compile time to persuade the compiler that all is well. At runtime the service loader execution path does not honour the add-exports requests to the service layer. > > If you use ?add-exports during java compilation, you almost certainly need to use it at runtime. That is, repeat your --add-export arguments to the java launcher. You may also need some --add-reads options. If you put you jars up somewhere, I can help find the right incantation if you?re still having trouble. > >> In the end my temporary solution was to add this stanza to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's constructor. >> >> --8<-- >> String backendFactoryClassName = System.getProperty("graal.backend.factory"); // org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >> if (backendFactoryClassName != null){ >> try { >> Class factoryClass = (Class) Class.forName(backendFactoryClassName); >> HotSpotBackendFactory factory = factoryClass.getDeclaredConstructor().newInstance(); >> Class arch = factory.getArchitecture(); >> HotSpotBackendFactory oldEntry = backends.put(arch, factory); >> }catch(Exception e){ >> >> } >> }else { >> //existing code >> } >> -->8-- >> >> Then I can build a jdk13 image and run using a combination of --add-modules along with -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >> >> Whilst this works for me. I am quite sure if our common goal is to get users to 'adopt/play-with' Graal, we can't expect them to build a dedicated OpenJDK image. >> >> I think it is an oversite to offer a ServiceProvider extension mechanism (HotSpotBackendFactory) only for the module system to forbid anyone from using it. > > This is a constraint imposed on us by the fact that Graal is part of the JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think the inconvenience of adding a few --add-exports and --add-reads options to javac and java is an acceptable overhead to experiment with Graal. We just need to get the incantation right and hope someone writes a nice blog article describing it ;-) > >> It would be great if we could offer a simpler mechanism (without resorting to --patch-module which does not seem to be supported by IntelliJ). >> >> We already have some protection by forcing -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just open this up a little more behind these current flags. > > As stated above, the JVMCI and Graal modules cannot be opened up by default. > > -Doug >> >> >> >> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon > wrote: >> Hi Gary, >> >>> On 20 Apr 2020, at 14:44, Gary Frost > wrote: >>> >>> Yeah I started created a module, but it seems for my module to compile I need jdk.internal.vm.ci module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my named module. Which seems wrong. >> >> Actually, that?s by design. Only standard JDK API can be exported unqualified from the JDK. Neither Graal nor JVMCI is public API and so --add-exports is needed for compilation. >> >>> >>> I wanted to avoid using GraalVM. I figured I should be able to do what I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >> >> You can - you just need --add-exports (and maybe some other options) to ?open up? the classes you?re trying to hook into. >> >>> I can 'get around it' by pulling the source for org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking this one file (to load my Factory) then building using using --patch-module jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is clearly hacky. I should not have to use --patch-module here. >>> >>> Alas using --patch-module renders IntelliJ next to useless (it ignores patch-module) as pointed out by John Rose and myself here. >>> https://youtrack.jetbrains.com/issue/IDEA-224342 >>> >>> Maybe the intent was that the JDK versions of Graal, were just to 'use Graal', not to allow developers to add there own features. >>> >>> Maybe that is the distiction between GraalVM and Graal enabled JDK. >> >> No, both of these don?t not offer open extension points. You need to force it open on the command as mentioned above. We can?t have just any old app code getting their hands on the compiler ;-) >> >> -Doug >> >>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon > wrote: >>> Hi Gary, >>> >>> Have you tried creating a module for your service? I think that?s required as of JDK 9. >>> >>> I suggest opening a PR on https://github.com/oracle/graal so we can have a more concrete discussion. >>> >>> -Doug >>> >>> > On 20 Apr 2020, at 11:17, Gary Frost > wrote: >>> > >>> > I feel I am missing something obvious ;) >>> > >>> > I am working (well playing actually) with Graal and JDK13 and have built my >>> > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>> > inject some custom code into generated x86. Very cool. >>> > >>> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory >>> > class (from JDK13 source) I can instantiate my factory directly, and I see >>> > my modified code. >>> > >>> > But of course I want to able to build against and use use my factory using >>> > a clean opendjdk JDK13 binary distribution. >>> > >>> > It looks like the >>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to >>> > allow me to offer my factory as a 'service provider' which the >>> > CompilerConfigurationFactory accesses via this call >>> > GraalServices.load(HotSpotBackendFactory.class) >>> > but nothing I try works. >>> > >>> > It looks like I need to provide a module-info.java class, but of course I >>> > only want my one service to override the default. >>> > >>> > Any suggestions/help oir pointers to docs would be appreciated. >>> > >>> > Gary >>> >> >> > From frost.gary at gmail.com Tue Apr 21 13:07:47 2020 From: frost.gary at gmail.com (Gary Frost) Date: Tue, 21 Apr 2020 14:07:47 +0100 Subject: BackEnd Service Provider. In-Reply-To: <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> Message-ID: Great, thanks for looking into it. Lets see what Alan makes of it. Gary On Tue, Apr 21, 2020 at 2:01 PM Doug Simon wrote: > Hi Gary, > > I cannot understand why --add-reads does not work and have reached out to > Alan Bateman for more insight. > > That said, good to know that at least --patch-module works, even if it > confuses IntelliJ. > > -Doug > > On 21 Apr 2020, at 14:37, Gary Frost wrote: > > I also added a makefile which uses patch-module which works. > > So > git clone https://github.com/grfrost/graal_service_issue.git > > cd graal_service_issue > # vi makefile* and set path to your JDK13 (JDK11 may work... I know some > API's have changed between 11 and 13) > make -f makefile-using-module-service-provider > make -f makefile-with-patch-module > > > On Tue, Apr 21, 2020 at 12:23 PM Gary Frost wrote: > >> Here is a public github project >> https://github.com/grfrost/graal_service_issue >> >> >> git clone https://github.com/grfrost/graal_service_issue.git >> >> cd graal_service_issue >> # vi Makefile and set path to your JDK13 (JDK11 may work... I know some >> API's have changed between 11 and 13) >> make >> >> Gary >> >> >> >> >> On Tue, Apr 21, 2020 at 11:59 AM Gary Frost wrote: >> >>> I apologise Doug >>> >>> My argument is that there is no combination of '-add-exports and >>> --add-reads options to javac and java' that will work here. Try it! ;) >>> >>> I do/did in fact have it at runtime as well. I stripped it from my >>> description because it seems to make no difference. >>> >>> /usr/lib/jvm/jdk-13/bin/java \ >>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >>> -XX:-TieredCompilation\ >>> >>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >>> >>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >>> --add-exports >>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >>> -p build -m org.grfstuff.compiler\ >>> -classpath build/org.grfstuff.violajones\ >>> org.grfstuff.violajones.ViolaJones >>> >>> I attached a zip of the project to my last email and it got stripped. >>> >>> My argument is that there is no combination of '-add-exports and >>> --add-reads options to javac and java' that will work. >>> >>> The only option is --patch-module which is not even officially offered >>> by javac or java and is not supported by IDE's >>> >>> $ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l >>> 0 >>> $ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l >>> 0 >>> >>> I only know of this option because I read the source code ;) >>> >>> Gary >>> >>> On Tue, Apr 21, 2020 at 11:47 AM Doug Simon >>> wrote: >>> >>>> >>>> >>>> On 21 Apr 2020, at 12:30, Gary Frost wrote: >>>> >>>> I don't want to flog this horse excessively, but please humour me one >>>> more time. >>>> >>>> Even after defining a module I can't offer my own service provider for >>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory >>>> >>>> I really think we need a way for folks to experiment with Graal >>>> backends without having to build there own JDK/JVM. >>>> >>>> I created a module to be able to map my service provider >>>> >>>> src/org.grfstuff.compiler/ >>>> ??? module-info.java >>>> ??? org >>>> ??? grfstuff >>>> ??? compiler >>>> ??? GrfStuffHotSpotBackendFactory.java >>>> >>>> >>>> Where my module-info.java is this. >>>> --8<-- >>>> module org.grfstuff.compiler { >>>> requires jdk.internal.vm.compiler; >>>> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with >>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory; >>>> } >>>> -->8-- >>>> >>>> My implementation is just enough code so I can see an instance of my >>>> backend get loaded. >>>> --8<-- >>>> package org.grfstuff.compiler; >>>> import org.graalvm.compiler.hotspot.HotSpotBackendFactory; >>>> import org.graalvm.compiler.serviceprovider.ServiceProvider; >>>> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory; >>>> >>>> @ServiceProvider(HotSpotBackendFactory.class) >>>> public class GrfStuffHotSpotBackendFactory extends >>>> AMD64HotSpotBackendFactory { >>>> public GrfStuffHotSpotBackendFactory(){ >>>> System.out.println("GrfStuffHotSpotFactory"); >>>> } >>>> public String toString() { >>>> return "GRFSTUFF_AMD64"; >>>> } >>>> } >>>> -->8-- >>>> >>>> I can build my module using appropriate --add-exports 'jiggery pokery ' >>>> :) >>>> >>>> javac -g \ >>>> -d build \ >>>> --add-exports >>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\ >>>> --add-exports >>>> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\ >>>> --add-exports >>>> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\ >>>> --module-source-path src\ >>>> --module org.grfstuff.compiler\ >>>> $(shell find src/org.grfstuff.compiler -name *.java) >>>> >>>> But at runtime (my app is a compute intensive voilajones face detection >>>> algorithm) >>>> java \ >>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >>>> -XX:-TieredCompilation\ >>>> >>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\ >>>> >>>> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\ >>>> -p build -m org.grfstuff.compiler\ >>>> -classpath build/org.grfstuff.violajones\ >>>> org.grfstuff.violajones.ViolaJones >>>> >>>> I get this error >>>> >>>> Error occurred during initialization of boot layer >>>> java.lang.module.ResolutionException: Module org.grfstuff.compiler does >>>> not read a module that exports org.graalvm.compiler.hotspot >>>> >>>> It would appear that whilst add-exports is enough at compile time to >>>> persuade the compiler that all is well. At runtime the service loader >>>> execution path does not honour the add-exports requests to the service >>>> layer. >>>> >>>> >>>> If you use ?add-exports during java compilation, you almost certainly >>>> need to use it at runtime. That is, repeat your --add-export arguments to >>>> the java launcher. You may also need some --add-reads options. If you put >>>> you jars up somewhere, I can help find the right incantation if you?re >>>> still having trouble. >>>> >>>> In the end my temporary solution was to add this stanza >>>> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's >>>> constructor. >>>> >>>> --8<-- >>>> String backendFactoryClassName = >>>> System.getProperty("graal.backend.factory"); // >>>> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >>>> if (backendFactoryClassName != null){ >>>> try { >>>> Class >>>> factoryClass = (Class) >>>> Class.forName(backendFactoryClassName); >>>> HotSpotBackendFactory factory = >>>> factoryClass.getDeclaredConstructor().newInstance(); >>>> Class arch = >>>> factory.getArchitecture(); >>>> HotSpotBackendFactory oldEntry = >>>> backends.put(arch, factory); >>>> }catch(Exception e){ >>>> >>>> } >>>> }else { >>>> //existing code >>>> } >>>> -->8-- >>>> >>>> Then I can build a jdk13 image and run using a combination of >>>> --add-modules along with >>>> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory >>>> >>>> Whilst this works for me. I am quite sure if our common goal is to get >>>> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated >>>> OpenJDK image. >>>> >>>> I think it is an oversite to offer a ServiceProvider extension >>>> mechanism (HotSpotBackendFactory) only for the module system to forbid >>>> anyone from using it. >>>> >>>> >>>> This is a constraint imposed on us by the fact that Graal is part of >>>> the JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I >>>> think the inconvenience of adding a few --add-exports and --add-reads >>>> options to javac and java is an acceptable overhead to experiment with >>>> Graal. We just need to get the incantation right and hope someone writes a >>>> nice blog article describing it ;-) >>>> >>>> It would be great if we could offer a simpler mechanism (without >>>> resorting to --patch-module which does not seem to be supported by >>>> IntelliJ). >>>> >>>> We already have some protection by forcing >>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just >>>> open this up a little more behind these current flags. >>>> >>>> >>>> As stated above, the JVMCI and Graal modules cannot be opened up by >>>> default. >>>> >>>> -Doug >>>> >>>> >>>> >>>> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon >>>> wrote: >>>> >>>>> Hi Gary, >>>>> >>>>> On 20 Apr 2020, at 14:44, Gary Frost wrote: >>>>> >>>>> Yeah I started created a module, but it seems for my module to compile >>>>> I need jdk.internal.vm.ci >>>>> >>>>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory >>>>> to my named module. Which seems wrong. >>>>> >>>>> >>>>> Actually, that?s by design. Only standard JDK API can be exported >>>>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so >>>>> --add-exports is needed for compilation. >>>>> >>>>> >>>>> I wanted to avoid using GraalVM. I figured I should be able to do >>>>> what I want, using stock/released JDK13 and >>>>> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler >>>>> >>>>> >>>>> You can - you just need --add-exports (and maybe some other options) >>>>> to ?open up? the classes you?re trying to hook into. >>>>> >>>>> I can 'get around it' by pulling the source for >>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking >>>>> this one file (to load my Factory) then building using >>>>> using --patch-module >>>>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is >>>>> clearly hacky. I should not have to use --patch-module here. >>>>> >>>>> Alas using --patch-module renders IntelliJ next to useless (it ignores >>>>> patch-module) as pointed out by John Rose and myself here. >>>>> https://youtrack.jetbrains.com/issue/IDEA-224342 >>>>> >>>>> >>>>> Maybe the intent was that the JDK versions of Graal, were just to 'use >>>>> Graal', not to allow developers to add there own features. >>>>> >>>>> Maybe that is the distiction between GraalVM and Graal enabled JDK. >>>>> >>>>> >>>>> No, both of these don?t not offer open extension points. You need to >>>>> force it open on the command as mentioned above. We can?t have just any old >>>>> app code getting their hands on the compiler ;-) >>>>> >>>>> -Doug >>>>> >>>>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon >>>>> wrote: >>>>> >>>>>> Hi Gary, >>>>>> >>>>>> Have you tried creating a module for your service? I think that?s >>>>>> required as of JDK 9. >>>>>> >>>>>> I suggest opening a PR on https://github.com/oracle/graal >>>>>> >>>>>> so we can have a more concrete discussion. >>>>>> >>>>>> -Doug >>>>>> >>>>>> > On 20 Apr 2020, at 11:17, Gary Frost wrote: >>>>>> > >>>>>> > I feel I am missing something obvious ;) >>>>>> > >>>>>> > I am working (well playing actually) with Graal and JDK13 and have >>>>>> built my >>>>>> > own implementation of >>>>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to >>>>>> > inject some custom code into generated x86. Very cool. >>>>>> > >>>>>> > If I hack the >>>>>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory >>>>>> > class (from JDK13 source) I can instantiate my factory directly, >>>>>> and I see >>>>>> > my modified code. >>>>>> > >>>>>> > But of course I want to able to build against and use use my >>>>>> factory using >>>>>> > a clean opendjdk JDK13 binary distribution. >>>>>> > >>>>>> > It looks like the >>>>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is >>>>>> setup to >>>>>> > allow me to offer my factory as a 'service provider' which the >>>>>> > CompilerConfigurationFactory accesses via this call >>>>>> > GraalServices.load(HotSpotBackendFactory.class) >>>>>> > but nothing I try works. >>>>>> > >>>>>> > It looks like I need to provide a module-info.java class, but of >>>>>> course I >>>>>> > only want my one service to override the default. >>>>>> > >>>>>> > Any suggestions/help oir pointers to docs would be appreciated. >>>>>> > >>>>>> > Gary >>>>>> >>>>>> >>>>> >>>> >>>> >>>> > From Alan.Bateman at oracle.com Wed Apr 22 07:12:38 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 22 Apr 2020 08:12:38 +0100 Subject: BackEnd Service Provider. In-Reply-To: <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> Message-ID: On 21/04/2020 14:01, Doug Simon wrote: > Hi Gary, > > I cannot understand why --add-reads does not work and have reached out > to Alan Bateman for more insight. > I read through the thread [1] and I don't see an obvious solution to this. The main issue here is that org.grfstuff.compiler wants to provide an implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory but that service type is not in org.grfstuff.compiler and org.graalvm.compiler.hotspot is not exported to org.grfstuff.compiler by any of the modules that it reads. The post resolution consistency check that it trips up on is specified in the Configuration API docs if you are interested. I assume `--add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler` was added in an attempt to workaround this but that CLI option is the equivalent of jdk.internal.vm.compiler invoking Module::addExports after the module graph has been reified. It doesn't impact resolution or any of the post resolution consistency checks at run-time. Yes, there is an unfortunate difference between between compile-time and run-time in this regard but that is something for another discussion. I don't think there are any obvious workarounds. The Module API defines addXXX methods to export or open packages at run-time, extend readability or add a service dependences but it doesn't define addProvides. I think it came up once during JDK 9 but more in the context of symmetry rather than a compelling use-case. It's not clear that the issue under discussion here is compelling enough, needs more thought as the scenario is a bit weird. If org.grfstuff.compiler was co developed with jdk.internal.vm.compiler then it could use a qualified export of course. Initially I thought you could workaround this by moving org.grfstuff.compiler? to the class path but this it's not going to work as ServiceLodaer usage in Graal seems to use module layers as congtext, so it will not find implementations on the class path. -Alan [1] https://mail.openjdk.java.net/pipermail/graal-dev/2020-April/005949.html From doug.simon at oracle.com Wed Apr 22 07:30:38 2020 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 22 Apr 2020 09:30:38 +0200 Subject: BackEnd Service Provider. In-Reply-To: References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> Message-ID: <72B844C8-AF90-42CB-83AE-E18EC3B4F1A8@oracle.com> Thanks for your valuable input Alan. So until the VM provides an --add-provides option (which is not currently planned), --patch-module is the only mechanism for dynamically adding providers of otherwise ?sealed? Graal services. We could indeed make Graal service loading look for providers on the class path but that would have to be off by default and thus require a command line option (e.g. -Dgraal.AllowServicesOnClassPath=true). However, that doesn?t seem like any less effort than having to use --patch-module. -Doug > On 22 Apr 2020, at 09:12, Alan Bateman wrote: > > On 21/04/2020 14:01, Doug Simon wrote: >> Hi Gary, >> >> I cannot understand why --add-reads does not work and have reached out to Alan Bateman for more insight. >> > I read through the thread [1] and I don't see an obvious solution to this. > > The main issue here is that org.grfstuff.compiler wants to provide an implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory but that service type is not in org.grfstuff.compiler and org.graalvm.compiler.hotspot is not exported to org.grfstuff.compiler by any of the modules that it reads. The post resolution consistency check that it trips up on is specified in the Configuration API docs if you are interested. > > I assume `--add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler` was added in an attempt to workaround this but that CLI option is the equivalent of jdk.internal.vm.compiler invoking Module::addExports after the module graph has been reified. It doesn't impact resolution or any of the post resolution consistency checks at run-time. Yes, there is an unfortunate difference between between compile-time and run-time in this regard but that is something for another discussion. > > I don't think there are any obvious workarounds. The Module API defines addXXX methods to export or open packages at run-time, extend readability or add a service dependences but it doesn't define addProvides. I think it came up once during JDK 9 but more in the context of symmetry rather than a compelling use-case. It's not clear that the issue under discussion here is compelling enough, needs more thought as the scenario is a bit weird. If org.grfstuff.compiler was co developed with jdk.internal.vm.compiler then it could use a qualified export of course. > > Initially I thought you could workaround this by moving org.grfstuff.compiler to the class path but this it's not going to work as ServiceLodaer usage in Graal seems to use module layers as congtext, so it will not find implementations on the class path. > > -Alan > > [1] https://mail.openjdk.java.net/pipermail/graal-dev/2020-April/005949.html From frost.gary at gmail.com Wed Apr 22 09:16:22 2020 From: frost.gary at gmail.com (Gary Frost) Date: Wed, 22 Apr 2020 10:16:22 +0100 Subject: BackEnd Service Provider. In-Reply-To: <72B844C8-AF90-42CB-83AE-E18EC3B4F1A8@oracle.com> References: <13EF9C67-5103-4CCA-8289-34FA0812C0DE@oracle.com> <59F3EC22-3039-4693-9533-EED43E74F425@oracle.com> <72452A53-8D80-4700-AD9B-F42FE3D1E637@oracle.com> <9DACFFE0-F6F9-4EC7-A72D-59201AFB2674@oracle.com> <72B844C8-AF90-42CB-83AE-E18EC3B4F1A8@oracle.com> Message-ID: Thanks for both looking into this I have an IntelliJ workaround that sort of works (I will pass on to John Rose, because he reported the same issue to IntelliJ), which is obvious in hindsight. I launch outside of IDE using --patch-module and get the process to wait, then attach from IDE. This gets around issue of IDE not honouring --patch-module. So I can at least debug in IDE. Just some more feedback on misspent hours ;) I actually modified JDK13 to allow me to pass in a property (name of factory implementing org.graalvm.compiler.hotspot.HotSpotBackendFactory) and built. Then of course the issues was that my factory was not in the classpath ;) I tried creating a classloader seeded via a jar name (via another property) but it all went downhill fairly fast... after that... curses were shouted... dogs were kicked and beer was consumed. So clearly --patch-module not only allows replacing 'system' classes, it allows all the other classes in the patched path, to be treated as if they were on system classpath, ala ye olde -Xbootclasspath. Anyway again, thanks for having a look, and possibly keeping this type of usecase in mind was we try to allow access to graal. It would be nice to make this easy for folk. PS. No dogs were really harmed, during the classloading hacking. Gary On Wed, Apr 22, 2020 at 8:31 AM Doug Simon wrote: > Thanks for your valuable input Alan. > > So until the VM provides an --add-provides option (which is not currently > planned), --patch-module is the only mechanism for dynamically adding > providers of otherwise ?sealed? Graal services. We could indeed make Graal > service loading look for providers on the class path but that would have to > be off by default and thus require a command line option (e.g. > -Dgraal.AllowServicesOnClassPath=true). However, that doesn?t seem like any > less effort than having to use --patch-module. > > -Doug > > > On 22 Apr 2020, at 09:12, Alan Bateman wrote: > > > > On 21/04/2020 14:01, Doug Simon wrote: > >> Hi Gary, > >> > >> I cannot understand why --add-reads does not work and have reached out > to Alan Bateman for more insight. > >> > > I read through the thread [1] and I don't see an obvious solution to > this. > > > > The main issue here is that org.grfstuff.compiler wants to provide an > implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory but > that service type is not in org.grfstuff.compiler and > org.graalvm.compiler.hotspot is not exported to org.grfstuff.compiler by > any of the modules that it reads. The post resolution consistency check > that it trips up on is specified in the Configuration API docs if you are > interested. > > > > I assume `--add-exports > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler` > was added in an attempt to workaround this but that CLI option is the > equivalent of jdk.internal.vm.compiler invoking Module::addExports after > the module graph has been reified. It doesn't impact resolution or any of > the post resolution consistency checks at run-time. Yes, there is an > unfortunate difference between between compile-time and run-time in this > regard but that is something for another discussion. > > > > I don't think there are any obvious workarounds. The Module API defines > addXXX methods to export or open packages at run-time, extend readability > or add a service dependences but it doesn't define addProvides. I think it > came up once during JDK 9 but more in the context of symmetry rather than a > compelling use-case. It's not clear that the issue under discussion here is > compelling enough, needs more thought as the scenario is a bit weird. If > org.grfstuff.compiler was co developed with jdk.internal.vm.compiler then > it could use a qualified export of course. > > > > Initially I thought you could workaround this by moving > org.grfstuff.compiler to the class path but this it's not going to work as > ServiceLodaer usage in Graal seems to use module layers as congtext, so it > will not find implementations on the class path. > > > > -Alan > > > > [1] > https://mail.openjdk.java.net/pipermail/graal-dev/2020-April/005949.html > > From jaroslav.tulach at oracle.com Thu Apr 23 12:00:41 2020 From: jaroslav.tulach at oracle.com (Jaroslav Tulach) Date: Thu, 23 Apr 2020 14:00:41 +0200 Subject: Debugging your Graal Compiler Extensions with GraalVM's IGV was: BackEnd Service Provider. In-Reply-To: References: <72B844C8-AF90-42CB-83AE-E18EC3B4F1A8@oracle.com> Message-ID: <3436173.W8mByuWhQQ@dellxps> Hello Gary. Here is a pull request that turns the project structure into a set of three Maven modules: https://github.com/grfrost/graal_service_issue/pull/1 As Maven is more approachable to common Java IDEs, one can use for example GraalVM's Ideal Graph Visualizer to edit, compile, run & debug the project. See the pull request for detailed instructions. Best regards. Jaroslav Tulach NetBeans Platform Architect OracleLabs Dne st?eda 22. dubna 2020 11:16:22 CEST, Gary Frost napsal(a): > Thanks for both looking into this > > I have an IntelliJ workaround that sort of works (I will pass on to John > Rose, because he reported the same issue to IntelliJ), which is obvious in > hindsight. I launch outside of IDE using --patch-module and get the > process to wait, then attach from IDE. This gets around issue of IDE not > honouring --patch-module. So I can at least debug in IDE. > > Just some more feedback on misspent hours ;) > > I actually modified JDK13 to allow me to pass in a property (name of > factory implementing org.graalvm.compiler.hotspot.HotSpotBackendFactory) > and built. Then of course the issues was that my factory was not in the > classpath ;) I tried creating a classloader seeded via a jar name (via > another property) but it all went downhill fairly fast... after that... > curses were shouted... dogs were kicked and beer was consumed. > > So clearly --patch-module not only allows replacing 'system' classes, it > allows all the other classes in the patched path, to be treated as if they > were on system classpath, ala ye olde -Xbootclasspath. > > Anyway again, thanks for having a look, and possibly keeping this type of > usecase in mind was we try to allow access to graal. It would be nice to > make this easy for folk. > > PS. No dogs were really harmed, during the classloading hacking. > > Gary > > On Wed, Apr 22, 2020 at 8:31 AM Doug Simon wrote: > > Thanks for your valuable input Alan. > > > > So until the VM provides an --add-provides option (which is not currently > > planned), --patch-module is the only mechanism for dynamically adding > > providers of otherwise ?sealed? Graal services. We could indeed make Graal > > service loading look for providers on the class path but that would have > > to > > be off by default and thus require a command line option (e.g. > > -Dgraal.AllowServicesOnClassPath=true). However, that doesn?t seem like > > any > > less effort than having to use --patch-module. > > > > -Doug > > > > > On 22 Apr 2020, at 09:12, Alan Bateman wrote: > > > > > > On 21/04/2020 14:01, Doug Simon wrote: > > >> Hi Gary, > > >> > > >> I cannot understand why --add-reads does not work and have reached out > > > > to Alan Bateman for more insight. > > > > > I read through the thread [1] and I don't see an obvious solution to > > > > this. > > > > > The main issue here is that org.grfstuff.compiler wants to provide an > > > > implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory but > > that service type is not in org.grfstuff.compiler and > > org.graalvm.compiler.hotspot is not exported to org.grfstuff.compiler by > > any of the modules that it reads. The post resolution consistency check > > that it trips up on is specified in the Configuration API docs if you are > > interested. > > > > > I assume `--add-exports > > > > jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compile > > r` was added in an attempt to workaround this but that CLI option is the > > equivalent of jdk.internal.vm.compiler invoking Module::addExports after > > the module graph has been reified. It doesn't impact resolution or any of > > the post resolution consistency checks at run-time. Yes, there is an > > unfortunate difference between between compile-time and run-time in this > > regard but that is something for another discussion. > > > > > I don't think there are any obvious workarounds. The Module API defines > > > > addXXX methods to export or open packages at run-time, extend readability > > or add a service dependences but it doesn't define addProvides. I think it > > came up once during JDK 9 but more in the context of symmetry rather than > > a > > compelling use-case. It's not clear that the issue under discussion here > > is > > compelling enough, needs more thought as the scenario is a bit weird. If > > org.grfstuff.compiler was co developed with jdk.internal.vm.compiler then > > it could use a qualified export of course. > > > > > Initially I thought you could workaround this by moving > > > > org.grfstuff.compiler to the class path but this it's not going to work > > as > > ServiceLodaer usage in Graal seems to use module layers as congtext, so it > > will not find implementations on the class path. > > > > > -Alan > > > > > > [1] > > > > https://mail.openjdk.java.net/pipermail/graal-dev/2020-April/005949.html From dean.long at oracle.com Thu Apr 23 23:48:06 2020 From: dean.long at oracle.com (Dean Long) Date: Thu, 23 Apr 2020 16:48:06 -0700 Subject: RFR(S) 8219607: Add support in Graal and AOT for hidden class Message-ID: https://bugs.openjdk.java.net/browse/JDK-8219607 http://cr.openjdk.java.net/~dlong/8219607/webrev/ This change adds support for the Class.isHidden() intrinsic to Graal. thanks, dl From vladimir.kozlov at oracle.com Fri Apr 24 00:57:04 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 23 Apr 2020 17:57:04 -0700 Subject: RFR(S) 8219607: Add support in Graal and AOT for hidden class In-Reply-To: References: Message-ID: <36d29a0e-e396-da7a-4945-ad9afb709b14@oracle.com> Hi Dean, Changes looks good. I see that compiler/graalunit/HotspotTest.java failed in tier1 (and tier3-graal). I assume it is 8243381. Thanks, Vladimir K On 4/23/20 4:48 PM, Dean Long wrote: > https://bugs.openjdk.java.net/browse/JDK-8219607 > http://cr.openjdk.java.net/~dlong/8219607/webrev/ > > This change adds support for the Class.isHidden() intrinsic to Graal. > > thanks, > > dl From dean.long at oracle.com Fri Apr 24 02:20:44 2020 From: dean.long at oracle.com (Dean Long) Date: Thu, 23 Apr 2020 19:20:44 -0700 Subject: RFR(S) 8219607: Add support in Graal and AOT for hidden class In-Reply-To: <36d29a0e-e396-da7a-4945-ad9afb709b14@oracle.com> References: <36d29a0e-e396-da7a-4945-ad9afb709b14@oracle.com> Message-ID: On 4/23/20 5:57 PM, Vladimir Kozlov wrote: > Hi Dean, > > Changes looks good. Thanks Vladimir. > > I see that compiler/graalunit/HotspotTest.java failed in tier1 (and > tier3-graal). I assume it is 8243381. Yes, I accidentally removed that sub-test from the problem list during testing, so it added some "noise" to the test results. dl > > Thanks, > Vladimir K > > On 4/23/20 4:48 PM, Dean Long wrote: >> https://bugs.openjdk.java.net/browse/JDK-8219607 >> http://cr.openjdk.java.net/~dlong/8219607/webrev/ >> >> This change adds support for the Class.isHidden() intrinsic to Graal. >> >> thanks, >> >> dl