BackEnd Service Provider.
Gary Frost
frost.gary at gmail.com
Tue Apr 21 10:59:03 UTC 2020
I apologise Doug
My argument is that there is no combination of '-add-exports and
--add-reads options to javac and java' that will work here. Try it! ;)
I do/did in fact have it at runtime as well. I stripped it from my
description because it seems to make no difference.
/usr/lib/jvm/jdk-13/bin/java \
-XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
-XX:-TieredCompilation\
-XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
-XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
--add-exports
jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
--add-exports
jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
--add-exports
jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
-p build -m org.grfstuff.compiler\
-classpath build/org.grfstuff.violajones\
org.grfstuff.violajones.ViolaJones
I attached a zip of the project to my last email and it got stripped.
My argument is that there is no combination of '-add-exports and
--add-reads options to javac and java' that will work.
The only option is --patch-module which is not even officially offered by
javac or java and is not supported by IDE's
$ /usr/lib/jvm/jdk-13/bin/javac --help | grep patch | wc -l
0
$ /usr/lib/jvm/jdk-13/bin/java --help | grep patch | wc -l
0
I only know of this option because I read the source code ;)
Gary
On Tue, Apr 21, 2020 at 11:47 AM Doug Simon <doug.simon at oracle.com> wrote:
>
>
> On 21 Apr 2020, at 12:30, Gary Frost <frost.gary at gmail.com> wrote:
>
> I don't want to flog this horse excessively, but please humour me one
> more time.
>
> Even after defining a module I can't offer my own service provider for
> org.graalvm.compiler.hotspot.HotSpotBackendFactory
>
> I really think we need a way for folks to experiment with Graal backends
> without having to build there own JDK/JVM.
>
> I created a module to be able to map my service provider
>
> src/org.grfstuff.compiler/
> ├── module-info.java
> └── org
> └── grfstuff
> └── compiler
> └── GrfStuffHotSpotBackendFactory.java
>
>
> Where my module-info.java is this.
> --8<--
> module org.grfstuff.compiler {
> requires jdk.internal.vm.compiler;
> provides org.graalvm.compiler.hotspot.HotSpotBackendFactory with
> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory;
> }
> -->8--
>
> My implementation is just enough code so I can see an instance of my
> backend get loaded.
> --8<--
> package org.grfstuff.compiler;
> import org.graalvm.compiler.hotspot.HotSpotBackendFactory;
> import org.graalvm.compiler.serviceprovider.ServiceProvider;
> import org.graalvm.compiler.hotspot.amd64.AMD64HotSpotBackendFactory;
>
> @ServiceProvider(HotSpotBackendFactory.class)
> public class GrfStuffHotSpotBackendFactory extends
> AMD64HotSpotBackendFactory {
> public GrfStuffHotSpotBackendFactory(){
> System.out.println("GrfStuffHotSpotFactory");
> }
> public String toString() {
> return "GRFSTUFF_AMD64";
> }
> }
> -->8--
>
> I can build my module using appropriate --add-exports 'jiggery pokery ' :)
>
> javac -g \
> -d build \
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot=org.grfstuff.compiler\
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.serviceprovider=org.grfstuff.compiler\
> --add-exports
> jdk.internal.vm.compiler/org.graalvm.compiler.hotspot.amd64=org.grfstuff.compiler\
> --module-source-path src\
> --module org.grfstuff.compiler\
> $(shell find src/org.grfstuff.compiler -name *.java)
>
> But at runtime (my app is a compute intensive voilajones face detection
> algorithm)
> java \
> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler
> -XX:-TieredCompilation\
>
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJones.compute\
>
> -XX:CompileCommand=compileonly,org/grfstuff/violajones/ViolaJOnes.run\
> -p build -m org.grfstuff.compiler\
> -classpath build/org.grfstuff.violajones\
> org.grfstuff.violajones.ViolaJones
>
> I get this error
>
> Error occurred during initialization of boot layer
> java.lang.module.ResolutionException: Module org.grfstuff.compiler does
> not read a module that exports org.graalvm.compiler.hotspot
>
> It would appear that whilst add-exports is enough at compile time to
> persuade the compiler that all is well. At runtime the service loader
> execution path does not honour the add-exports requests to the service
> layer.
>
>
> If you use —add-exports during java compilation, you almost certainly need
> to use it at runtime. That is, repeat your --add-export arguments to the
> java launcher. You may also need some --add-reads options. If you put you
> jars up somewhere, I can help find the right incantation if you’re still
> having trouble.
>
> In the end my temporary solution was to add this stanza
> to org.graalvm.compiler.hotspot.CompilerConfigurationFactory.DefaultBackendMap's
> constructor.
>
> --8<--
> String backendFactoryClassName =
> System.getProperty("graal.backend.factory"); //
> org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
> if (backendFactoryClassName != null){
> try {
> Class<? extends HotSpotBackendFactory> factoryClass =
> (Class<? extends HotSpotBackendFactory>)
> Class.forName(backendFactoryClassName);
> HotSpotBackendFactory factory =
> factoryClass.getDeclaredConstructor().newInstance();
> Class<? extends Architecture> arch =
> factory.getArchitecture();
> HotSpotBackendFactory oldEntry =
> backends.put(arch, factory);
> }catch(Exception e){
>
> }
> }else {
> //existing code
> }
> -->8--
>
> Then I can build a jdk13 image and run using a combination of
> --add-modules along with
> -Dgraal.backend.factory=org.grfstuff.compiler.GrfStuffHotSpotBackendFactory
>
> Whilst this works for me. I am quite sure if our common goal is to get
> users to 'adopt/play-with' Graal, we can't expect them to build a dedicated
> OpenJDK image.
>
> I think it is an oversite to offer a ServiceProvider extension mechanism
> (HotSpotBackendFactory) only for the module system to forbid anyone from
> using it.
>
>
> This is a constraint imposed on us by the fact that Graal is part of the
> JDK. Just like Unsafe, JVMCI and Graal cannot be publicly exported. I think
> the inconvenience of adding a few --add-exports and --add-reads options to
> javac and java is an acceptable overhead to experiment with Graal. We just
> need to get the incantation right and hope someone writes a nice blog
> article describing it ;-)
>
> It would be great if we could offer a simpler mechanism (without resorting
> to --patch-module which does not seem to be supported by IntelliJ).
>
> We already have some protection by forcing
> -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler maybe we can just
> open this up a little more behind these current flags.
>
>
> As stated above, the JVMCI and Graal modules cannot be opened up by
> default.
>
> -Doug
>
>
>
> On Mon, Apr 20, 2020 at 1:52 PM Doug Simon <doug.simon at oracle.com> wrote:
>
>> Hi Gary,
>>
>> On 20 Apr 2020, at 14:44, Gary Frost <frost.gary at gmail.com> wrote:
>>
>> Yeah I started created a module, but it seems for my module to compile I
>> need jdk.internal.vm.ci
>> <https://urldefense.com/v3/__http://jdk.internal.vm.ci__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPPY4GJB0$>
>> module to export org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>> my named module. Which seems wrong.
>>
>>
>> Actually, that’s by design. Only standard JDK API can be exported
>> unqualified from the JDK. Neither Graal nor JVMCI is public API and so
>> --add-exports is needed for compilation.
>>
>>
>> I wanted to avoid using GraalVM. I figured I should be able to do what I
>> want, using stock/released JDK13 and -XX:+UnlockExperimentalVMOptions
>> -XX:+UseJVMCICompiler
>>
>>
>> You can - you just need --add-exports (and maybe some other options) to
>> “open up” the classes you’re trying to hook into.
>>
>> I can 'get around it' by pulling the source for
>> org.graalvm.compiler.hotspot.CompilerConfigurationFactory locally. Hacking
>> this one file (to load my Factory) then building using
>> using --patch-module
>> jdk.internal.vm.compiler=src/jdk.internal.vm.compiler/java, but this is
>> clearly hacky. I should not have to use --patch-module here.
>>
>> Alas using --patch-module renders IntelliJ next to useless (it ignores
>> patch-module) as pointed out by John Rose and myself here.
>> https://youtrack.jetbrains.com/issue/IDEA-224342
>> <https://urldefense.com/v3/__https://youtrack.jetbrains.com/issue/IDEA-224342__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPKLKl6Nu$>
>>
>> Maybe the intent was that the JDK versions of Graal, were just to 'use
>> Graal', not to allow developers to add there own features.
>>
>> Maybe that is the distiction between GraalVM and Graal enabled JDK.
>>
>>
>> No, both of these don’t not offer open extension points. You need to
>> force it open on the command as mentioned above. We can’t have just any old
>> app code getting their hands on the compiler ;-)
>>
>> -Doug
>>
>> On Mon, Apr 20, 2020 at 11:54 AM Doug Simon <doug.simon at oracle.com>
>> wrote:
>>
>>> Hi Gary,
>>>
>>> Have you tried creating a module for your service? I think that’s
>>> required as of JDK 9.
>>>
>>> I suggest opening a PR on https://github.com/oracle/graal
>>> <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!IQVX1iggkDspLlkVBXNQ7IulcpTkED_VhK6SjF2apyR4-XW-E7oZHqeQPMQSQQsk$>
>>> so we can have a more concrete discussion.
>>>
>>> -Doug
>>>
>>> > On 20 Apr 2020, at 11:17, Gary Frost <frost.gary at gmail.com> wrote:
>>> >
>>> > I feel I am missing something obvious ;)
>>> >
>>> > I am working (well playing actually) with Graal and JDK13 and have
>>> built my
>>> > own implementation of
>>> org.graalvm.compiler.hotspot.HotSpotBackendFactory to
>>> > inject some custom code into generated x86. Very cool.
>>> >
>>> > If I hack the org.graalvm.compiler.hotspot.CompilerConfigurationFactory
>>> > class (from JDK13 source) I can instantiate my factory directly, and
>>> I see
>>> > my modified code.
>>> >
>>> > But of course I want to able to build against and use use my factory
>>> using
>>> > a clean opendjdk JDK13 binary distribution.
>>> >
>>> > It looks like the
>>> > class org.graalvm.compiler.hotspot.CompilerConfigurationFactory is
>>> setup to
>>> > allow me to offer my factory as a 'service provider' which the
>>> > CompilerConfigurationFactory accesses via this call
>>> > GraalServices.load(HotSpotBackendFactory.class)
>>> > but nothing I try works.
>>> >
>>> > It looks like I need to provide a module-info.java class, but of
>>> course I
>>> > only want my one service to override the default.
>>> >
>>> > Any suggestions/help oir pointers to docs would be appreciated.
>>> >
>>> > Gary
>>>
>>>
>> <graal_service_issue.zip>
>
>
>
More information about the graal-dev
mailing list