BackEnd Service Provider.
Doug Simon
doug.simon at oracle.com
Tue Apr 21 10:47:27 UTC 2020
> On 21 Apr 2020, at 12:30, Gary Frost <frost.gary at gmail.com> wrote:
>
> I don't want to flog this horse excessively, but please humour me one more time.
>
> Even after defining a module I can't offer my own service provider for org.graalvm.compiler.hotspot.HotSpotBackendFactory
>
> I really think we need a way for folks to experiment with Graal backends without having to build there own JDK/JVM.
>
> I created a module to be able to map my service provider
>
> src/org.grfstuff.compiler/
> ├── module-info.java
> └── org
> └── grfstuff
> └── compiler
> └── GrfStuffHotSpotBackendFactory.java
>
>
> Where my module-info.java is this.
> --8<--
> module org.grfstuff.compiler {
> requires jdk.internal.vm.compiler;
> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with org.grfstuff.compiler.GrfStuffHotSpotBackendFactory;
> }
> -->8--
>
> My implementation is just enough code so I can see an instance of my backend get loaded.
> --8<--
> package org.grfstuff.compiler;
> import org.graalvm.compiler.hotspot.HotSpotBackendFactory;
> import org.graalvm.compiler.serviceprovider.ServiceProvider;
> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory;
>
> @ServiceProvider(HotSpotBackendFactory.class)
> public class GrfStuffHotSpotBackendFactory extends AMD64HotSpotBackendFactory {
> public GrfStuffHotSpotBackendFactory(){
> System.out.println("GrfStuffHotSpotFactory");
> }
> public String toString() {
> return "GRFSTUFF_AMD64";
> }
> }
> -->8--
>
> I can build my module using appropriate --add-exports 'jiggery pokery ' :)
>
> javac -g \
> -d build \
> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
> --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
> --module-source-path src\
> --module org.grfstuff.compiler\
> $(shell find src/org.grfstuff.compiler -name *.java)
>
> But at runtime (my app is a compute intensive voilajones face detection algorithm)
> java \
> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -XX:-TieredCompilation\
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
> -p build -m org.grfstuff.compiler\
> -classpath build/org.grfstuff.violajones\
> org.grfstuff.violajones.ViolaJones
>
> I get this error
>
> Error occurred during initialization of boot layer
> java.lang.module.ResolutionException: Module org.grfstuff.compiler does not read a module that exports org.graalvm.compiler.hotspot
>
> It would appear that whilst add-exports is enough at compile time to persuade the compiler that all is well. At runtime the service loader execution path does not honour the add-exports requests to the service layer.
If you use —add-exports during java compilation, you almost certainly need to use it at runtime. That is, repeat your --add-export arguments to the java launcher. You may also need some --add-reads options. If you put you jars up somewhere, I can help find the right incantation if you’re still having trouble.
> In the end my temporary solution was to add this stanza to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's constructor.
>
> --8<--
> String backendFactoryClassName = System.getProperty("graal.backend.factory"); // org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
> if (backendFactoryClassName != null){
> try {
> Class<? extends HotSpotBackendFactory> factoryClass = (Class<? extends HotSpotBackendFactory>) Class.forName(backendFactoryClassName);
> HotSpotBackendFactory factory = factoryClass.getDeclaredConstructor().newInstance();
> Class<? extends Architecture> arch = factory.getArchitecture();
> HotSpotBackendFactory oldEntry = backends.put(arch, factory);
> }catch(Exception e){
>
> }
> }else {
> //existing code
> }
> -->8--
>
> Then I can build a jdk13 image and run using a combination of --add-modules along with -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>
> Whilst this works for me. I am quite sure if our common goal is to get users to 'adopt/play-with' Graal, we can't expect them to build a dedicated OpenJDK image.
>
> I think it is an oversite to offer a ServiceProvider extension mechanism (HotSpotBackendFactory) only for the module system to forbid anyone from using it.
This is a constraint imposed on us by the fact that Graal is part of the JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think the inconvenience of adding a few --add-exports and --add-reads options to javac and java is an acceptable overhead to experiment with Graal. We just need to get the incantation right and hope someone writes a nice blog article describing it ;-)
> It would be great if we could offer a simpler mechanism (without resorting to --patch-module which does not seem to be supported by IntelliJ).
>
> We already have some protection by forcing -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just open this up a little more behind these current flags.
As stated above, the JVMCI and Graal modules cannot be opened up by default.
-Doug
>
>
>
> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon <doug.simon at oracle.com <mailto:doug.simon at oracle.com>> wrote:
> Hi Gary,
>
>> On 20 Apr 2020, at 14:44, Gary Frost <frost.gary at gmail.com <mailto:frost.gary at gmail.com>> wrote:
>>
>> Yeah I started created a module, but it seems for my module to compile I need jdk.internal.vm.ci <https://urldefense.com/v3/__http://jdk.internal.vm.ci__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPPY4GJB0$> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to my named module. Which seems wrong.
>
> Actually, that’s by design. Only standard JDK API can be exported unqualified from the JDK. Neither Graal nor JVMCI is public API and so --add-exports is needed for compilation.
>
>>
>> I wanted to avoid using GraalVM. I figured I should be able to do what I want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
>
> You can - you just need --add-exports (and maybe some other options) to “open up” the classes you’re trying to hook into.
>
>> I can 'get around it' by pulling the source for org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking this one file (to load my Factory) then building using using --patch-module jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is clearly hacky. I should not have to use --patch-module here.
>>
>> Alas using --patch-module renders IntelliJ next to useless (it ignores patch-module) as pointed out by John Rose and myself here.
>> https://youtrack.jetbrains.com/issue/IDEA-224342 <https://urldefense.com/v3/__https://youtrack.jetbrains.com/issue/IDEA-224342__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPKLKl6Nu$>
>>
>> Maybe the intent was that the JDK versions of Graal, were just to 'use Graal', not to allow developers to add there own features.
>>
>> Maybe that is the distiction between GraalVM and Graal enabled JDK.
>
> No, both of these don’t not offer open extension points. You need to force it open on the command as mentioned above. We can’t have just any old app code getting their hands on the compiler ;-)
>
> -Doug
>
>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon <doug.simon at oracle.com <mailto:doug.simon at oracle.com>> wrote:
>> Hi Gary,
>>
>> Have you tried creating a module for your service? I think that’s required as of JDK 9.
>>
>> I suggest opening a PR on https://github.com/oracle/graal <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPMQSQQsk$> so we can have a more concrete discussion.
>>
>> -Doug
>>
>> > On 20 Apr 2020, at 11:17, Gary Frost <frost.gary at gmail.com <mailto:frost.gary at gmail.com>> wrote:
>> >
>> > I feel I am missing something obvious ;)
>> >
>> > I am working (well playing actually) with Graal and JDK13 and have built my
>> > own implementation of org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>> > inject some custom code into generated x86. Very cool.
>> >
>> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory
>> > class (from JDK13 source) I can instantiate my factory directly, and I see
>> > my modified code.
>> >
>> > But of course I want to able to build against and use use my factory using
>> > a clean opendjdk JDK13 binary distribution.
>> >
>> > It looks like the
>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is setup to
>> > allow me to offer my factory as a 'service provider' which the
>> > CompilerConfigurationFactory accesses via this call
>> > GraalServices.load(HotSpotBackendFactory.class)
>> > but nothing I try works.
>> >
>> > It looks like I need to provide a module-info.java class, but of course I
>> > only want my one service to override the default.
>> >
>> > Any suggestions/help oir pointers to docs would be appreciated.
>> >
>> > Gary
>>
>
> <graal_service_issue.zip>
More information about the graal-dev
mailing list